Zipf Plot Research Articles

Zipf's law applications in patent landscape analysis

New business insights are shown to be extractable from patent landscapes by the mathematical method of discrete Pareto analysis. By applying to patent publication distributions, a method analogous to that proposed by the linguist George Kingsley Zipf, metrics and methods of visualization are introduced which quantify scale, dominance and consolidation of a patent landscape. The key results of the method are illustrated in the Zipf plot of assignee patent publication count versus assignee rank for the lithography patent landscape shown below.

The new wealth of nations: How STEM fields generate the prosperity and inequality of individuals, companies, and countries

Fundamental research in physics has long been a prerequisite for computer scientists and engineers to design innovative products, such as laptops and cell phones. Technological innovations and fundamental research are both part of the so-called STEM fields (Science, Technology, Engineering, and Mathematics), which are known to substantially contribute to economic growth. However, the questions still remain: how much contribution do these fields make to both wealth accumulation and inequality at different levels of analysis? First, analyzing the lists of world’s wealthiest individuals, the Zipf plot analysis demonstrates that STEM billionaires contribute more to wealth inequality than their non-STEM counterparts. Analyzing the companies in the S&P500, we find that STEM firms contribute more to wealth inequality and have larger growth rates on average than the non-STEM firms. Finally, we show that the more STEM graduates in a country, the larger its GDP growth rate. In combination, we demonstrate that STEM is a fractal mechanism that drives wealth accumulation—and the wealth inequality— at different scales of economy—from individual wealth to firm valuation to country GDP.

The Gender Productivity Gap in Croatian Science: Women Are Catching up with Males and Becoming Even Better.

How much different genders contribute to citations and whether we see different gender patterns between STEM and non-STEM researchers are questions that have long been studied in academia. Here we analyze the research output in terms of citations collected from the Web of Science of males and females from the largest Croatian university, University of Zagreb. Applying the Mann–Whitney statistical test, for most faculties, we demonstrate no gender difference in research output except for seven faculties, where males are significantly better than females on six faculties. We find that female STEM full professors are significantly more cited than male colleagues, while male non-STEM assistant professors are significantly more cited than their female colleagues. There are ten faculties where females have the larger average citations than their male colleagues and eleven faculties where the most cited researcher is woman. For the most cited researchers, our Zipf plot analyses demonstrate that both genders follow power laws, where the exponent calculated for male researchers is moderately larger than the exponent for females. The exponent for STEM citations is slightly larger than the exponent obtained for non-STEM citations, implying that compared to non-STEM, STEM research output leads to fatter tails and so larger citations inequality than non-STEM.

Characterization of Strong Motion Generation Regions of Earthquake Slip Using Extreme Value Theory

This article characterizes the regions of earthquake slip that are responsible for strong motion generation (SMG), using extreme value theory (EVT). A three-step iterative procedure involving Zipf plot, mean excess function plot and moment ratio plot is used to define the threshold value for a slip model. This threshold slip demarcates the region of SMG in the slip model. The iterative procedure ensures that slip values above the threshold slip follow generalized Pareto distribution (GPD). The regions where slip is greater than the corresponding threshold slip are defined as regions of SMG. A total of 159 slip samples filtered from the SRCMOD catalogue are analyzed in this paper. In order to identify the region of SMG, the first step is to obtain effective slip dimensions. A novel method based on strong motion duration (SMD) is proposed to obtain the effective dimensions of slip model. In the next step, the iterative procedure is employed to obtain threshold slip and the regions of SMG for all the slip samples. The corresponding area of SMG is estimated from the number of sub-faults in the region of SMG and the sub-fault dimensions. Regression analysis is carried out to establish source scaling relationships for the rupture parameters with respect to seismic moment. The considered rupture parameters are effective length, effective width, effective area, effective mean slip, threshold slip and area of SMG of the slip model. On a logarithmic scale, these rupture parameters are observed to follow a linearly increasing trend with the seismic moment. The proposed equations can be used to estimate the rupture parameters for future earthquake events.

Concept detection using text exemplars aligned with a specialized ontology

Knowledge extraction from text documents requires identifying and classifying semantic content. Utilizing an appropriate domain ontology can facilitate this process if words and phrases can be linked to the classes and relationships within the ontology. This paper presents an exemplar-based algorithm to link text to semantically similar classes within an ontology constructed for the chronic pain medicine domain. Human annotators linked classes to text segments within a random document set for construction of an exemplar dictionary, which we examined for completeness using Zipf plot analysis. An algorithm was created to use this dictionary on previously unseen text to form a map between sentence text and probable class assignments. We performed a 5×5 cross-validation between human and algorithm annotations and examined both ROC and precision versus recall curves to show that the algorithm can identify the many medical and biopsychosocial components from the texts. We briefly describe a use case for detecting pain relief from various interventions utilizing the word-by-class maps. We conclude that an exemplar-based method can be a valuable tool in knowledge extraction from texts that share similar construction, such as medical progress notes.

Rank distributions: Frequency vs. magnitude.

We examine the relationship between two different types of ranked data, frequencies and magnitudes. We consider data that can be sorted out either way, through numbers of occurrences or size of the measures, as it is the case, say, of moon craters, earthquakes, billionaires, etc. We indicate that these two types of distributions are functional inverses of each other, and specify this link, first in terms of the assumed parent probability distribution that generates the data samples, and then in terms of an analog (deterministic) nonlinear iterated map that reproduces them. For the particular case of hyperbolic decay with rank the distributions are identical, that is, the classical Zipf plot, a pure power law. But their difference is largest when one displays logarithmic decay and its counterpart shows the inverse exponential decay, as it is the case of Benford law, or viceversa. For all intermediate decay rates generic differences appear not only between the power-law exponents for the midway rank decline but also for small and large rank. We extend the theoretical framework to include thermodynamic and statistical-mechanical concepts, such as entropies and configuration.

Are your data really Pareto distributed?

Pareto distributions, and power laws in general, have demonstrated to be very useful models to describe very different phenomena, from physics to finance. In recent years, the econophysical literature has proposed a large amount of papers and models justifying the presence of power laws in economic data.Most of the times, this Paretianity is inferred from the observation of some plots, such as the Zipf plot and the mean excess plot. If the Zipf plot looks almost linear, then everything is ok and the parameters of the Pareto distribution are estimated. Often with OLS.Unfortunately, as we show in this paper, these heuristic graphical tools are not reliable. To be more exact, we show that only a combination of plots can give some degree of confidence about the real presence of Paretianity in the data.We start by reviewing some of the most important plots, discussing their points of strength and weakness, and then we propose some additional tools that can be used to refine the analysis.

The effect of nonstationarity on models inferred from neural data

Neurons subject to a common nonstationary input may exhibit a correlated firing behavior. Correlations in the statistics of neural spike trains also arise as the effect of interaction between neurons. Here we show that these two situations can be distinguished with machine learning techniques, provided that the data are rich enough. In order to do this, we study the problem of inferring a kinetic Ising model, stationary or nonstationary, from the available data. We apply the inference procedure to two data sets: one from salamander retinal ganglion cells and the other from a realistic computational cortical network model. We show that many aspects of the concerted activity of the salamander retinal neurons can be traced simply to the external input. A model of non-interacting neurons subject to a nonstationary external field outperforms a model with stationary input with couplings between neurons, even accounting for the differences in the number of model parameters. When couplings are added to the nonstationary model, for the retinal data, little is gained: the inferred couplings are generally not significant. Likewise, the distribution of the sizes of sets of neurons that spike simultaneously and the frequency of spike patterns as a function of their rank (Zipf plots) are well explained by an independent-neuron model with time-dependent external input, and adding connections to such a model does not offer significant improvement. For the cortical model data, robust couplings, well correlated with the real connections, can be inferred using the nonstationary model. Adding connections to this model slightly improves the agreement with the data for the probability of synchronous spikes but hardly affects the Zipf plot.

Deviation of Zipf's and Heaps' Laws in Human Languages with Limited Dictionary Sizes

Zipf's law on word frequency and Heaps' law on the growth of distinct words are observed in Indo-European language family, but it does not hold for languages like Chinese, Japanese and Korean. These languages consist of characters, and are of very limited dictionary sizes. Extensive experiments show that: (i) The character frequency distribution follows a power law with exponent close to one, at which the corresponding Zipf's exponent diverges. Indeed, the character frequency decays exponentially in the Zipf's plot. (ii) The number of distinct characters grows with the text length in three stages: It grows linearly in the beginning, then turns to a logarithmical form, and eventually saturates. A theoretical model for writing process is proposed, which embodies the rich-get-richer mechanism and the effects of limited dictionary size. Experiments, simulations and analytical solutions agree well with each other. This work refines the understanding about Zipf's and Heaps' laws in human language systems.

RMT Assessments of the Market Latent Information Embedded in the Stocks′ Raw, Normalized, and Partial Correlations

We present here assessment of the latent market information embedded in the raw, affinity (normalized), and partial correlations. We compared the Zipf plot, spectrum, and distribution of the eigenvalues for each matrix with the results of the corresponding random matrix. The analysis was performed on stocks belonging to the New York and Tel Aviv Stock Exchange, for the time period of January 2000 to March 2009. Our results show that in comparison to the raw correlations, the affinity matrices highlight the dominant factors of the system, and the partial correlation matrices contain more information. We propose that significant stock market information, which cannot be captured by the raw correlations, is embedded in the affinity and partial correlations. Our results further demonstrate the differences between NY and TA markets.

Data Analysis and Statistical Behaviors of Stock Market Fluctuations

In this paper, the data of Chinese stock markets is analyzed by the statistical methods and computer sciences. The fluctuations of stock prices and trade volumes are investigated by the method of Zipf plot, where Zipf plot technique is frequently used in physics science. In the first part of the present paper, the data of stocks prices and trade volumes in Shanghai Stock Exchange and Shenzhen Stock Exchange is analyzed, the statistical behaviors of stocks prices and trade volumes are studied. We select the daily data for Chinese stock markets during the years 2002-2006, by analyzing the data, we discuss the statistical properties of fat tails phenomena and the power law distributions for the daily stocks prices and trade volumes. In the second part, we consider the fat ails phenomena and the power law distributions of Shanghai Stock Exchange Index and Shenzhen Stock Exchange Index during the years 2002-2007, and we also compare the distributions of these two indices with the corresponding distributions of the Zipf plot.

Korean Family Name Distribution in the Past

We empirically study the genealogical trees of ten families for about five centuries in Korea. Although each family tree contains only the paternal part, the family names of women married to the family have been recorded, which allows us to estimate roughly the family name distributions for the past five hundred years. Revealed is the fact that the unique Korean family name distribution, characterized by a logarithmic form of the cumulative distribution and an exponentially decaying rank-size plot often called the Zipf plot, has remained unchanged for a long time. We discuss the implications of our results within a recently suggested theoretical model and compare them with observations in other countries in which power-law forms are abundantly found.

Gradually truncated log-normal in USA publicly traded firm size distribution

We study the statistical distribution of firm size for USA and Brazilian publicly traded firms through the Zipf plot technique. Sale size is used to measure firm size. The Brazilian firm size distribution is given by a log-normal distribution without any adjustable parameter. However, we also need to consider different parameters of log-normal distribution for the largest firms in the distribution, which are mostly foreign firms. The log-normal distribution has to be gradually truncated after a certain critical value for USA firms. Therefore, the original hypothesis of proportional effect proposed by Gibrat is valid with some modification for very large firms. We also consider the possible mechanisms behind this distribution.

Estimation of the extreme-value index and generalized quantile plots

In extreme-value analysis, a central topic is the adaptive estimation of the extreme-value index γ. Hitherto, most of the attention in this area has been devoted to the case γ>0, that is, when F ¯ is a regularly varying function with index -1/γ. In addition to the well-known Hill estimator, many other estimators are currently available. Among the most important are the kernel-type estimators and the weighted least-squares slope estimators based on the Pareto quantile plot or the Zipf plot, as reviewed by Csorgo and Viharos. Using an exponential regression model (ERM) for spacings between successive extreme order statistics, both Beirlant et al. and Feuerverger and Hall introduced bias-reduced estimators.

Statistical properties of nucleotides in human chromosomes 21 and 22

In this paper the statistical properties of nucleotides in human chromosomes 21 and 22 are investigated. The n -tuple Zipf analysis with n = 3, 4, 5, 6, and 7 is used in our investigation. It is found that the most common n -tuples are those which consist only of adenine (A) and thymine (T), and the rarest n -tuples are those in which GC or CG pattern appears twice. With the n -tuples become more and more frequent, the double GC or CG pattern becomes a single GC or CG pattern. The percentage of four nucleotides in the rarest ten and the most common ten n -tuples are also considered in human chromosomes 21 and 22, and different behaviors are found in the percentage of four nucleotides. Frequency of appearance of n -tuple f ( r ) as a function of rank r is also examined. We find the n -tuple Zipf plot shows a power-law behavior for r < 4 n −1 and a rapid decrease for r > 4 n −1 . In order to explore the interior statistical properties of human chromosomes 21 and 22 in detail, we divide the chromosome sequence into some moving windows and we discuss the percentage of ξη ( ξ , η = A, C, G, T) pair in those moving windows. In some particular regions, there are some obvious changes in the percentage of ξη pair, and there maybe exist functional differences. The normalized number of repeats N 0 ( l ) can be described by a power law: N 0 ( l ) ∼ l − μ . The distance distributions P 0 ( S ) between two nucleotides in human chromosomes 21 and 22 are also discussed. A two-order polynomial fit exists in those distance distributions: log P 0 ( S ) = a + bS + cS 2 , and it is quite different from the random sequence.

Information Theory Applied to Animal Communication Systems and Its Possible Application to SETI

Information theory, as first introduced by Claude Shannon (Shannon &amp; Weaver 1949) quantitatively evaluates the organizational complexity of communication systems. At the same time George Zipf was examining linguistic structure in a way that was mathematically similar to the components of the Shannon first-order entropy (Zipf 1949). Both Shannon's and Zipf's mathematical procedures have been applied to animal communication and recently have been providing insightful results. The Zipf plot is a useful tool for a first estimate of the characterization of a communication system's complexity (which can later be examined for complex structure at deeper levels using Shannon entropic analysis). In this paper we shall discuss some of the applications and pitfalls of using the Zipf distribution as a preliminary evaluator of the communication complexity of a signaling system.

Scaling and universality in the micro-structure of urban space

We present a broad, phenomenological picture of the distribution of the length of open space linear segments, l, derived from maps of 36 cities in 14 different countries. By scaling the Zipf plot of l, we obtain two master curves for a sample of cities, which are not a function of city size. We show that a third class of cities is not easily classifiable into these two universality classes. The cumulative distribution of l displays power-law tails with two distinct exponents, α B =2 and α R =3. We suggest a link between our data and the possibility of observing and modelling urban geometric structures using Lévy processes.

Power-law distribution of city fires

This paper presents results of fire distributions in China and Switzerland. The frequency-size (loss) distribution and the rank-size distribution (or Zipf plot) of city fires are plotted to check their fit with a power-law relationship. The results indicate that fire data in both countries have good power-law distributions and the power-law relation is invariant with scale and time, i.e. fires in different places or in different periods will have the same distribution characteristics. Steady power-law distribution corresponds with self-organized criticality (SOC) and is expected to provide practical implications for fire protection.

How popular is your paper? An empirical study of the citation distribution

Numerical data for the distribution of citations are examined for: (i) papers published in 1981 in journals which are catalogued by the Institute for Scientific Information (783,339 papers) and (ii) 20 years of publications in Physical Review D, vols. 11-50 (24,296 papers). A Zipf plot of the number of citations to a given paper versus its citation rank appears to be consistent with a power-law dependence for leading rank papers, with exponent close to -1/2. This, in turn, suggests that the number of papers with x citations, N(x), has a large-x power law decay N(x)~x^{-alpha}, with alpha approximately equal to 3.

Zipf and the size distribution of firms

Based on a sample of 50441 independent UK companies of all sizes covering all sectors of the UK economy, the size distribution of companies is close to lognormal, even though the hypothesis of lognormality can be rejected statistically. A Zipf plot shows that the Pareto distribution is a poor fit to the upper 5% tail.