Word Frequency Distributions Research Articles

The state of ambidexterity research: a data mining approach

PurposeThe academic discipline of ambidexterity is recognized as an established field within strategic management and organization theory. Extant works in the field recognize the recent burgeoning academic emphasis on ambidexterity and suggest either an impending focus or a decline of the field. Although there have been attempts to review the field of ambidexterity, most of these reviews have not followed a systematic and scientific approach for extricating the key themes emerging within. The purpose of the study is to inductively and systematically capture the main elements constituting the current boundary of the field.Design/methodology/approachTo achieve the stated purpose, this study uses data/text mining to analyze abstracts of scholarly ambidexterity papers published over the past two decades (1997-2016) in academic journals. Using text-mining, lexical analysis is performed to compute the frequency distribution of words. The frequently occurring words are studied in detail and their association with ambidexterity is also studied by means of co-relation. This analysis (used for generation of first order themes) is then complemented by a manual analysis of each first order theme to come up with trends and sub-themes lying within.FindingsThe analysis extracts eight distinct themes that indicate the current boundary of ambidexterity research. The findings highlight the potential areas for future academic attention such as networks, business models, leadership, dynamic capability and their inter-linkages with the field of ambidexterity. Overall, the field of ambidexterity is receiving heightened academic interest coupled with a dynamic proliferation across a host of related management fields. Apart from the required future consolidation, the field also needs new insights to enhance its explanatory power.Research limitations/implicationsThis study analyzes abstracts of 504 works on ambidexterity (in the last two decades) to inductively indicate the current boundaries of the field using a data/text mining approach. Papers that do not explicitly mention ambidexterity in their abstracts, title or keywords are not included in the analysis.Practical implicationsThe insights of the analysis will not only help researchers but also offer practitioners a good view point about the myriad of paths (not restricted to contextual, structural and temporal) through which ambidexterity can be promoted within and at the organization level.Originality/valueThe three-fold contribution of this study is a systematic and scientific approach adopted to define the current boundary of the field of ambidexterity, followed by an exploration of a set of eight distinct themes and finally the identification of ongoing debates, research gaps and future research questions in light of the analysis performed.

Bayesian analysis of Word frequency distribution in context of Indian literature

Bayesian analysis of Word frequency distribution in context of Indian literature

Quantitative Aspects of PDTB-Style Discourse Relations across Languages

Frequency distribution of words, syntax and semantics in many languages abides by certain laws. However, because of the shortage of discourse corpora, few studies have examined whether the frequency of discourse relations follows some distributional patterns. Although there is some research based on the Rhetorical Structure Theory discourse treebank (RST-DT), each of these studies is limited to a single language. Otherwise to the RST-DT, the Penn Discourse Treebank (PDTB), adopting another annotation system, has had an enormous influence on the study of discourse structure and discourse annotation. Discourse corpora in other languages, such as Chinese, Hindi, Turkish, Czech and Arabic have been annotated following PDTB style. With the data from these discourse treebanks, we find that the rank-frequency of discourse relations follow the same pattern and that these languages share significant similarities in using semantic relations to organize the discourse. It is evidenced in our research that humans assume the relationship between two consecutive sentences is a causal connection or expansion link for fewer connectives used, but the relation of contrast is the most marked by connectives. This research will be of significance for understanding the homogeneity of discourse structure across languages.

Tracking word semantic change in biomedical literature

Up to this point, research on written scholarly communication has focused primarily on syntactic, rather than semantic, analyses. Consequently, we have yet to understand semantic change as it applies to disciplinary discourse. The objective of this study is to illustrate word semantic change in biomedical literature. To that end, we identify a set of representative words in biomedical literature based on word frequency and word-topic probability distributions. A word2vec language model is then applied to the identified words in order to measure word- and topic-level semantic changes. We find that for the selected words in PubMed, overall, meanings are becoming more stable in the 2000s than they were in the 1980s and 1990s. At the topic level, the global distance of most topics (19 out of 20 tested) is declining, suggesting that the words used to discuss these topics are stabilizing semantically. Similarly, the local distance of most topics (19 out of 20) is also declining, showing that the meanings of words from these topics are becoming more consistent with those of their semantic neighbors. At the word level, this paper identifies two different trends in word semantics, as measured by the aforementioned distance metrics: on the one hand, words can form clusters with their semantic neighbors, and these words, as a cluster, coevolve semantically; on the other hand, words can drift apart from their semantic neighbors while nonetheless stabilizing in the global context. In relating our work to language laws on semantic change, we find no overwhelming evidence to support either the law of parallel change or the law of conformity.

The Diary of Boima Kiakpomgbo from Mando Town (Liberia): A Quantitative Study of a Vai Text

We analyse a Liberian text from the first quarter of the twentieth century written in Vai, a Mande language of West Africa, using an indigenous syllabic script. Rank–frequency distributions of syllabic symbols and words are studied. The distribution of symbols is well-fitted by the 1-displaced negative hypergeometric distribution, confirming previous studies on alphabetic scripts, with a similar interpretation of parameters. Two regimes are discovered for the saturation of symbols in the text (growth of the number of types approaching the complete set versus the number of tokens). For the distribution of words, the Zipf–Mandelbrot law is found to be a proper model rather than the simple Zipfian dependence.

The dependence of frequency distributions on multiple meanings of words, codes and signs

The dependence of the frequency distributions due to multiple meanings of words in a text is investigated by deleting letters. By coding the words with fewer letters the number of meanings per coded word increases. This increase is measured and used as an input in a predictive theory. For a text written in English, the word-frequency distribution is broad and fat-tailed, whereas if the words are only represented by their first letter the distribution becomes exponential. Both distribution are well predicted by the theory, as is the whole sequence obtained by consecutively representing the words by the first L=6,5,4,3,2,1 letters. Comparisons of texts written by Chinese characters and the same texts written by letter-codes are made and the similarity of the corresponding frequency-distributions are interpreted as a consequence of the multiple meanings of Chinese characters. This further implies that the difference of the shape for word-frequencies for an English text written by letters and a Chinese text written by Chinese characters is due to the coding and not to the language per se.

Dependence of exponents on text length versus finite-size scaling for word-frequency distributions.

Some authors have recently argued that a finite-size scaling law for the text-length dependence of word-frequency distributions cannot be conceptually valid. Here we give solid quantitative evidence for the validity of this scaling law, using both careful statistical tests and analytical arguments based on the generalized central-limit theorem applied to the moments of the distribution (and obtaining a novel derivation of Heaps' law as a by-product). We also find that the picture of word-frequency distributions with power-law exponents that decrease with text length [X. Yan and P. Minnhagen, Physica A 444, 828 (2016)PHYADX0378-437110.1016/j.physa.2015.10.082] does not stand with rigorous statistical analysis. Instead, we show that the distributions are perfectly described by power-law tails with stable exponents, whose values are close to 2, in agreement with the classical Zipf's law. Some misconceptions about scaling are also clarified.

LDA*

We present LDA*, a system that has been deployed in one of the largest Internet companies to fulfil their requirements of "topic modeling as an internal service" ---relying on thousands of machines, engineers in different sectors submit their data, some are as large as 1.8TB, to LDA* and get results back in hours. LDA* is motivated by the observation that none of the existing topic modeling systems is robust enough ---Each of these existing systems is designed for a specific point in the tradeoff space that can be sub-optimal, sometimes by up to 10×, across workloads. Our first contribution is a systematic study of all recently proposed samplers: AliasLDA, F+LDA, LightLDA, and WarpLDA. We discovered a novel system tradeoff among these samplers. Each sampler has different sampling complexity and performs differently, sometimes by 5×, on documents with different lengths. Based on this tradeoff, we further developed a hybrid sampler that uses different samplers for different types of documents. This hybrid approach works across a wide range of workloads and outperforms the fastest sampler by up to 2x. We then focused on distributed environments in which thousands of workers, each with different performance (due to virtualization and resource sharing), coordinate to train a topic model. Our second contribution is an asymmetric parameter server architecture that pushes some computation to the parameter server side. This architecture is motivated by the skew of the word frequency distribution and a novel tradeoff we discovered between communication and computation. With this architecture, we outperform the traditional, symmetric architecture by up to 2×. With these two contributions, together with a carefully engineered implementation, our system is able to outperform existing systems by up to 10× and has already been running to provide topic modeling services for more than six months.

Content Analysis: Frequency Distribution of Words

Wide range of academic fields use content analysis. In the core of most common content analysis lies frequency distribution of individual words. Web sites and documents are mined for usage of certain words as well as for their frequency. We introduce a user written command wordfreq to process content (on-line and local) and to prepare a frequency distribution of individual words. Additionally, another user written command wordcloud is introduced to draw a simple word cloud graph for visual analysis of the frequent usage of specific words.

Statistical Probability Distributions of the Noble Quran

Classification of raw data through a frequency distribution is very important to determine the nature of the data distribution (probability distribution) and their structures. This paper describes the classifications of the number of verses (Ayas), words and letters in the chapters (Sura) of the Noble Quran using frequency distributions (tables) and then drawing histograms for each of the three types in order to find their structures. The main objective of this paper is to test and compare the frequency distributions of verses, words and letters of the Noble Quran with the common theoretical probability distributions using appropriate statistical methods. The results showed that the probability distributions for the number of verses and words per Quranic chapter are Exponential and Gamma respectively.

Variation in Word Frequency Distributions: Definitions, Measures and Implications for a Corpus-Based Language Typology

Word frequencies are central to linguistic studies investigating processing difficulty, learnability, age of acquisition, diachronic transmission and the relative weight given to a concept in society. However, there are few cross-linguistic studies on entire distributions of word frequencies, and even less on systematic changes within them. Here, we first define and test an exact measure for the relative difference between distributions – the Normalised Frequency Difference (NFD). We then apply this measure to parallel corpora in overall 19 languages, explaining systematic variation in the frequency distributions within the same language and across different languages. We further establish the NFD between lemmatised and un-lemmatised corpora as a frequency-based measure of inflectional productivity of a language. Finally, we argue that quantitative measures like the NFD can advance language typology beyond abstract, theory-driven expert judgments, towards more corpus-based, empirical and reproducible analyses.

Zipf's Law in Aphasia Across Languages: A Comparison of English, Hungarian and Greek

We investigated Zipf’s law in fluent and non-fluent aphasics’ spontaneous speech in English, Hungarian, and Greek. A previous study showed that the word frequency distribution in Dutch non-fluent aphasic speech conforms to Zipf’s law, although with a different slope. In this project we investigated to what extent these results can be generalized to other languages and to fluent aphasic speech. The results suggest that both the fluent and the non-fluent aphasic speech of English, Hungarian and Greek conform to Zipf’s law, and that differences in slope can be related to a language’s morphological properties and a group’s particular language impairments.

Coverage-based frequency bands: A proposal

Second language vocabulary research makes much use of word frequency lists and their division into bands. In recent years, bands of 1,000 items have become conventional. However, there does not seem to be any firm basis or rationale for this. Conventional banding may be questioned since the utility of words varies greatly depending on frequency, because there are enormous differences in frequency within higher bands, and because the reliability of the placement of words in bands becomes progressively poorer at lower frequency levels. This article suggests an alternative approach: basing bands on coverage levels. Because of the frequency distribution of words, this means the highest frequency bands would contain very few words, while lower frequency bands would contain a great many words. The article shows how such bands can be constructed and presents a re-analysis of the results of a vocabulary test designed with conventional bands in terms of coverage-based bands. This re-analysis produces a very different profile of learners’ knowledge, and it is argued that the shape of this profile may be more useful in terms of guiding instruction in that it gives a clearer indication of which words should be targeted for a group of learners. It is further argued that the smaller number of words contained in coverage-based bands at higher frequency levels makes them a more feasible basis for instruction. The article thus concludes that coverage-based bands may be a fruitful avenue for researchers to explore.

Generalized entropies and the similarity of texts

We show how generalized Gibbs–Shannon entropies can provide new insights on the statistical properties of texts. The universal distribution of word frequencies (Zipf’s law) implies that the generalized entropies, computed at the word level, are dominated by words in a specific range of frequencies. Here we show that this is the case not only for the generalized entropies but also for the generalized (Jensen–Shannon) divergences, used to compute the similarity between different texts. This finding allows us to identify the contribution of specific words (and word frequencies) for the different generalized entropies and also to estimate the size of the databases needed to obtain a reliable estimation of the divergences. We test our results in large databases of books (from the google n-gram database) and scientific papers (indexed by Web of Science).

Statistical Analysis of Word Frequency Distribution in Lithuanian Texts of Different Genres

The paper examines Lithuanian texts of different authors and genres. The main points ofinterest – the number of words, the number of different words and word frequencies. Structural type distributionand Zipf’s law are applied for describing the frequency distribution of words in the text. It is obvious that thelexical diversity of any text can be defined by different words that are used in the text, also called vocabulary.It is shown that the information contained in a reduced vocabulary is enough for dividing the texts analyzedin this article into groups by genre and author using a hierarchical clustering method. In this case, distancesbetween clusters are measured using the Jaccard distance measure, and clusters are aggregated using the Wardmethod.

A high‐capacity and secure least significant bit embedding approach based on word and letter frequencies

AbstractIn this paper, we propose a novel least significant bit embedding approach that capitalizes on the skewed distribution of letter and word frequencies to achieve higher image capacity, quality, and security. We initially conduct a study that involves all of the character frequencies using a data set of 14.245 billion characters. Huffman coding for each character is generated on the basis of its probability of occurrence. Furthermore, the top 100 000 most frequent words are transformed into a smaller ciphertext that has a lower cost. Our work demonstrates that recognizing characters and words on the basis of their frequency patterns and prioritizing them accordingly has a greater prospect of reducing the overall cost of embedding. The proposed scheme significantly outperforms Lempel–Ziv–Welch compression with an average of 45% fewer embedded bits. Moreover, the image quality is improved by a mean peak signal‐to‐noise ratio value of 6.9%. The proposed method also establishes a security embedding by proposing a novel shuffling algorithm. Copyright © 2017 John Wiley & Sons, Ltd.

Measuring information-based energy and temperature of literary texts

Abstract We apply a statistical method, information-based energy, to quantify informative symbolic sequences. To apply this method to literary texts, it is assumed that different words with different occurrence frequencies are at different energy levels, and that the energy-occurrence frequency distribution obeys a Boltzmann distribution. The temperature within the Boltzmann distribution can be an indicator for the author’s writing capacity as the repertory of thoughts. The relative temperature of a text is obtained by comparing the energy-occurrence frequency distributions of words collected from one text versus from all texts of the same author. Combining the relative temperature with the Shannon entropy as the text complexity, the information-based energy of the text is defined and can be viewed as a quantitative evaluation of an author’s writing performance. We demonstrate the method by analyzing two authors, Shakespeare in English and Jin Yong in Chinese, and find that their well-known works are associated with higher information-based energies. This method can be used to measure the creativity level of a writer’s work in linguistics, and can also quantify symbolic sequences in different systems.

Pow law in random symbolic sequences

Zipf’s law is an important linguistics law. In 1940s, Zipf found power law in the distribution of the word frequencies. The word rankings are in descending order of their occurrence frequencies. Through a series of experiments of random input letter sequences, the results show that the power law distribution has existed in not only natural language utterances but also random symbolic sequences.

TextGen: a realistic text data content generation method for modern storage system benchmarks

Modern storage systems incorporate data compressors to improve their performance and capacity. As a result, data content can significantly influence the result of a storage system benchmark. Because real-world proprietary datasets are too large to be copied onto a test storage system, and most data cannot be shared due to privacy issues, a benchmark needs to generate data synthetically. To ensure that the result is accurate, it is necessary to generate data content based on the characterization of real-world data properties that influence the storage system performance during the execution of a benchmark. The existing approach, called SDGen, cannot guarantee that the benchmark result is accurate in storage systems that have built-in word-based compressors. The reason is that SDGen characterizes the properties that influence compression performance only at the byte level, and no properties are characterized at the word level. To address this problem, we present TextGen, a realistic text data content generation method for modern storage system benchmarks. TextGen builds the word corpus by segmenting real-world text datasets, and creates a word-frequency distribution by counting each word in the corpus. To improve data generation performance, the word-frequency distribution is fitted to a lognormal distribution by maximum likelihood estimation. The Monte Carlo approach is used to generate synthetic data. The running time of TextGen generation depends only on the expected data size, which means that the time complexity of TextGen is O(n). To evaluate TextGen, four real-world datasets were used to perform an experiment. The experimental results show that, compared with SDGen, the compression performance and compression ratio of the datasets generated by TextGen deviate less from real-world datasets when end-tagged dense code, a representative of word-based compressors, is evaluated.

Phylogenetic tree construction using trinucleotide usage profile (TUP).

BackgroundIt has been a challenging task to build a genome-wide phylogenetic tree for a large group of species containing a large number of genes with long nucleotides sequences. The most popular method, called feature frequency profile (FFP-k), finds the frequency distribution for all words of certain length k over the whole genome sequence using (overlapping) windows of the same length. For a satisfactory result, the recommended word length (k) ranges from 6 to 15 and it may not be a multiple of 3 (codon length). The total number of possible words needed for FFP-k can range from 46=4096 to 415.ResultsWe propose a simple improvement over the popular FFP method using only a typical word length of 3. A new method, called Trinucleotide Usage Profile (TUP), is proposed based only on the (relative) frequency distribution using non-overlapping windows of length 3. The total number of possible words needed for TUP is 43=64, which is much less than the total count for the recommended optimal “resolution” for FFP. To build a phylogenetic tree, we propose first representing each of the species by a TUP vector and then using an appropriate distance measure between pairs of the TUP vectors for the tree construction. In particular, we propose summarizing a DNA sequence by a matrix of three rows corresponding to three reading frames, recording the frequency distribution of the non-overlapping words of length 3 in each of the reading frame. We also provide a numerical measure for comparing trees constructed with various methods.ConclusionsCompared to the FFP method, our empirical study showed that the proposed TUP method is more capable of building phylogenetic trees with a stronger biological support. We further provide some justifications on this from the information theory viewpoint. Unlike the FFP method, the TUP method takes the advantage that the starting of the first reading frame is (usually) known. Without this information, the FFP method could only rely on the frequency distribution of overlapping words, which is the average (or mixture) of the frequency distributions of three possible reading frames. Consequently, we show (from the entropy viewpoint) that the FFP procedure could dilute important gene information and therefore provides less accurate classification.