First-order Stationary Markov Research Articles

Robust parametric inference for finite Markov chains

We consider the problem of statistical inference in a parametric finite Markov chain model and develop a robust estimator of the parameters defining the transition probabilities via minimization of a suitable (empirical) version of the popular density power divergence. Based on a long sequence of observations from a first-order stationary Markov chain, we have defined the minimum density power divergence estimator (MDPDE) of the underlying parameter and rigorously derived its asymptotic and robustness properties under appropriate conditions. Performance of the MDPDEs is illustrated theoretically as well as empirically for some common examples of finite Markov chain models. Its applications in robust testing of statistical hypotheses are also discussed along with (parametric) comparison of two Markov chain sequences. Several directions for extending the MDPDE and related inference are also briefly discussed for multiple sequences of Markov chains, higher order Markov chains and non-stationary Markov chains with time-dependent transition probabilities. Finally, our proposal is applied to analyze corporate credit rating migration data of three international markets.

On Existence of Common Malicious System Call Codes in Android Malware Families

Most of the existing Android malware detection mechanisms are based on machine learning algorithms. The problem with the machine learning approaches is the difficulty in finding the best features that uniquely characterize the malwares. Hence, in this article we explore the property that uniquely characterizes Android malware applications. Toward this, we model the system call sequence generated by a malware application as a stationary first-order ergodic Markov chain and prove the existence of typical patterns which contains the malicious system call code of the application. In our implementation, we find the occurrence of common malicious system call codes in the system call sequence of several malware families. Finally a malware detection mechanism is proposed based on the occurrence of malicious system call code s in the system call sequence of an application. We obtain a consistent accuracy of around 0.95 in balanced, slightly unbalanced, and highly unbalanced data sets. In the balanced and slightly unbalanced data sets we obtain greater precision than 0.90; whereas in the highly unbalanced data sets the precision obtained are slightly lower at 0.72.

Sequence‐Oriented Stochastic Model of RO‐TRNGs for Entropy Evaluation

Ring oscillator-based true random number generators (RO-TRNGs) are widely used to generate unpredictable random numbers for cryptographic systems. Entropy is usually adopted to quantitatively measure the unpredictability of a TRNG. There have been several stochastic models such as the time-oriented and phaseoriented ones built to evaluate the entropy of ElementaryRO-TRNGs with single oscillator. However, these models are not suitable for the TRNGs composed of multiple oscillators (Multiple-RO-TRNGs), which can obtain more randomness and higher throughput. Considering this, we propose a sequence-oriented stochastic model for the entropy evaluation of RO-TRNGs, named the first-order stationary Markov source model. This model is extensible for the Multiple-RO-TRNGs. Based on that, we present a detailed method to determine the entropy of Multiple-ROTRNGs. Our proposed model is verified by experiments. Besides, our method can also be a guide to design ROTRNGs with both high entropy and high throughput.

Spatiotemporal methods for analysis of urban system dynamics: an application to Chile

This paper presents a methodological procedure to evaluate the influence of spatial proximity on evolution of cities to detect regional differences in their spatiotemporal dynamics. The six-step method based on a set of statistical methods can be computed with a new R package: estdaR. The first step consists of the usual characterization of the cross-sectional distribution of the urban areas by means of nonparametric estimations of density functions for a set of significant years. In the second and third steps, the growth process is modeled as a first-order stationary Markov chain to evaluate the effect of global and local spatial autocorrelation on the transition probabilities with a set of indices based on the spatial version of the standard Markov chain. The fourth, fifth, and sixth steps perform in-depth analysis to detect the existence and interaction of spatial regimes in the movement direction and ranking mobility of urban distribution. We apply this novel strategy for the period 1930–2002 to analyze the entire Chilean urban system—not only the Central Zone, in which most of the population and economic activities are concentrated, but also other urban zones in the country.

Joint Separable and Non-Separable Transforms for Next-Generation Video Coding.

Throughout the past few decades, the separable Discrete Cosine Transform (DCT), particularly the DCT type II, has been widely used in image and video compression. It is well known that, under first-order stationary Markov conditions, DCT is an efficient approximation of the optimal Karhunen-Loève transform. However, for natural image and video sources, the adaptivity of a single separable transform with fixed core is rather limited for the highly dynamic image statistics, e.g., textures and arbitrarily directed edges. It is also known that non-separable transforms can achieve better compression efficiency for images with directional texture patterns, yet they are computationally complex, especially when the transform size is large. In order to achieve higher transform coding gains with relatively low-complexity implementations, we propose a joint separable and non-separable transform. The proposed separable primary transform, named Enhanced Multiple Transform (EMT), applies multiple transform cores from a pre-defined subset of sinusoidal transforms, and the transform selection is signaled in a joint block level manner. Moreover, a Non-Separable Secondary Transform (NSST) method is proposed to operate in conjunction with EMT. Unlike the existing non-separable transform schemes which require excessive amounts of memory and computation, the proposed NSST efficiently improves coding gain with much lower complexity. Extensive experimental results show that the proposed methods, in a state-of-the-art video codec, such as HEVC, can provide significant coding gains (average 6.9% and 4.5% bitrate reductions for intra and random-access coding, respectively).

Bayesian Gaussian Mixture Linear Inversion for Geophysical Inverse Problems

A Bayesian linear inversion methodology based on Gaussian mixture models and its application to geophysical inverse problems are presented in this paper. The proposed inverse method is based on a Bayesian approach under the assumptions of a Gaussian mixture random field for the prior model and a Gaussian linear likelihood function. The model for the latent discrete variable is defined to be a stationary first-order Markov chain. In this approach, a recursive exact solution to an approximation of the posterior distribution of the inverse problem is proposed. A Markov chain Monte Carlo algorithm can be used to efficiently simulate realizations from the correct posterior model. Two inversion studies based on real well log data are presented, and the main results are the posterior distributions of the reservoir properties of interest, the corresponding predictions and prediction intervals, and a set of conditional realizations. The first application is a seismic inversion study for the prediction of lithological facies, P- and S-impedance, where an improvement of 30% in the root-mean-square error of the predictions compared to the traditional Gaussian inversion is obtained. The second application is a rock physics inversion study for the prediction of lithological facies, porosity, and clay volume, where predictions slightly improve compared to the Gaussian inversion approach.

Is gold an inflation-hedge? Evidence from an interrupted Markov-switching cointegration model

This paper investigates the inflation hedging role of gold price after controlling for the prices of other investment assets. We use annual data on the U.S. economy spanning from 1833 to 2013. We employ a recently developed flexible nonlinear approach that allows for potential ‘interruption’ in the long run equilibrium relationship in which the equilibrium term dynamics is modelled as an AR(1) depending upon an unobserved state process that is a stationary first-order Markov chain in two states, stationarity and non-stationarity. While, a battery of standard cointegration tests without and with breaks could not find evidence to support the inflation hedging role of gold, results from the flexible nonlinear approach indicate the existence of temporary cointegration between gold price and inflation during 1864, 1919, 1932, 1934, 1976, 1980 and 1982. The interruptions in the long-run relationship at different time periods seem to be associated with the different structural changes that affected the gold market.

Modeling the Customer Satisfaction Influence on the Long Term Sales: Example with Leading OTC Analgesics INN on National Market

Background: The customer satisfaction models are used to examine brand loyalty and sales. The utilization of the counter medicines depends directly on the level of knowledge of consumers, preferences and their satisfaction could be considered as an important predictor for their revenue. Objectives: The goal of the current study is to develop a Markov model for assessing the influence of the customer satisfaction on long term sales of leading OTC international nonproprietary names (INNs) of analgesics on the national market. Methods: Two first-order stationary Markov models based on marketing data for OTC analgesics sales and customer satisfaction inquiry, particularly from metamizole (MET), paracetamol (PAR), acetysal (ASA), and ibuprofen (IBU) were created and manipulated. The first model considered the very satisfied customers and the second the very satisfied and the somewhat satisfied customers. Results: MET is the INN with the most loyal customers followed by PAR. The product Markov matrix was derived after multiplications of the matrixes with market share and loyal customers’ probabilities. The steady state is achieved after 17 years for the group of satisfied customers and after 40 iterations for the group of somewhat satisfied. The market fluctuations are more dynamic in the second model probably due to lower determination of customers purchasing behavior. Conclusions: The model allows prediction of the long term changes in sales, differences between the groups of customers and long term marketing fluctuations. It could be useful in companies’ strategic sales management.

Data-Driven Tracking Based on Kalman Filter

A good model of the target will extract useful information about the target’s state from observations effectively. There are many models used to maneuvering target tracking, such as constant-velocity (CV) models, Singer acceleration model (zero-mean first-order Markov model) and “current” model (Mean-Adaptive Acceleration Model), etc. While due to the complexity of maneuvering target, to seek the target model which can get better performance is still a subject worthy of study. For the AR process, autocorrelation function is estimated by the random sampling points in this paper. We have the statistics relation between the autocorrelation function and variance based on a first-order stationary Markov process. Then the system parameters are obtained and a model is developed based on statistics relation, which needn’t set unknown parameter. Simulation shows the model developed can adaptively get the model parameter and obtain good performance.

Modeling the role of rainfall patterns in seasonal malaria transmission

Seasonal total precipitation is well known to affect malaria transmission because Anopheles mosquitoes depend on standing water for breeding habitat. However, the within-season temporal pattern of the rainfall influences persistence of standing water and thus rainfall patterns can also affect mosquito population dynamics in water-limited environments. Here, using a numerical simulation, I show that intraseasonal rainfall pattern accounts for 39% of the variance in simulated mosquito abundance in a Niger Sahel village where malaria is endemic but highly seasonal. I apply a field validated coupled hydrology and entomology model. Using synthetic rainfall time series generated using a stationary first-order Markov Chain model, I hold all variables except hourly rainfall constant, thus isolating the contribution of rainfall pattern to variance in mosquito abundance. I further show the utility of hydrology modeling using topography to assess precipitation effects by analyzing collected water. Time-integrated surface area of pools explains 70% of the variance in simulated mosquito abundance from a mechanistic model, and time-integrated surface area of pools persisting longer than 7 days explains 82% of the variance. Correlations using the hydrology model output explain more variance in mosquito abundance than the 60% from rainfall totals. I extend this analysis to investigate the impacts of this effect on malaria vector mosquito populations under climate shift scenarios, holding all climate variables except precipitation constant. In these scenarios, rainfall mean and variance change with climatic change, and the modeling approach evaluates the impact of non-stationarity in rainfall and the associated rainfall patterns on expected mosquito activity.

An Error Resilient Video Coding Scheme Using Embedded Wyner–Ziv Description With Decoder Side Non-Stationary Distortion Modeling

In this paper, we propose a generic error resilient video coding (ERVC) scheme using embedded Wyner-Ziv (WZ) description. At the encoder side, a joint source-channel R-D optimized mode selection (JSC-RDO-MS) algorithm with WZ-coded anchor frames is statistically studied and developed. Given a stationary first-order Markov Gaussian source, the proposed mode optimization is justified by an analysis of the RD impact on the WZ bit-rate. JSC-RDO-MS involves in the estimation of expected rate and distortion of WZ coding with the unavailable side information, and the WZ bit-rate of each coding mode is determined based on the error correction capability of the specific WZ codec. At the decoder side, an online correlation noise model between the source and the side-information is proposed with a mixture of Laplacians whose parameters are attained to reflect the coherence of the motion field of successive frames and the energy of prediction residual. Each mixture component represents the statistical distribution of prediction residuals, and the mixing coefficients represent the amount of errors in motion compensation. The proposed scheme achieves the so-called classification gain by exploiting the spatially non-stationary characteristics of the motion field and texture. Extensive experimental results show that the proposed WZ-ERVC scheme achieves a better overall RD performance than existing ERVC schemes, and the proposed modeling algorithm also significantly outperforms the conventional Laplacian model by up to 2 dB.

In Search of “Better-than-DCT” Unitary Transforms for Encoding of Residual Signals

It is well known that the discrete cosine transform (DCT) closely approximates the optimal Karhunen-Loève transform (KLT) under the first-order stationary Markov condition with a strong inter-pixel correlation. However, if the inter-pixel correlation is weak or becomes negative, transforms other than the DCT would possibly become better. In this letter, we present a design framework for finding new unitary transforms according to two principles: 1) they are indeed more efficient than the DCT in encoding of signals with a weak or negative inter-pixel correlation (e.g., the residual signals after motion-compensation or intra- prediction) and 2) each has a fixed transform matrix (i.e., signal-independent) and can be implemented as efficiently as the DCT. Some of the new transforms will be used to demonstrate an improved R-D performance in the video coding scenario.

Synthesis of reduced Kalman filter with the guaranteed estimation quality of dynamic system state

Synthesis of a suboptimal reduced Kalman filter with the guaranteed estimation quality of a state of dynamic system subject to nonwhite noise disturbances is considered. A calculation model containing additional white noise disturbances is used for selection of parameters of such a filter. Intensity of the added disturbances is chosen in such a way that the calculation covariance matrix, which is found as the solution of the Riccati equation in the filter covariance channel, is larger than the real covariance matrix of the error estimate \( \hat X \)0(t) in the quadratic form inequality sense. In this case, it is possible to consider this matrix as a measure of accuracy of the performed estimation of a dynamic system state. An example of the filter synthesis for description of disturbances with stationary first-order Markov processes is given.

Spatial analysis of urban growth in Spain, 1900–2001

The purpose of this paper is to improve the knowledge of the Spanish urban system. We study the evolution of population growth among the group of 722 municipalities included in the Spanish urban areas over the period 1900–2001. A spatial SUR model is estimated for Zipf’s law and shows the existence of two main phases: divergence (1900–1980) and convergence (1980–2001). Then, the cross-sectional distribution of urban population is characterized by means of nonparametric estimations of density functions and the growth process is modeled as a first-order stationary Markov chain. Spatial effects are finally introduced within the Markov chain framework using regional conditioning. This analysis shows a low interclass mobility, i.e., a high-persistence of urban municipalities to stay in their own class from one decade to another over the whole period, and the influence of the geographical environment on urban population dynamism.

Asymptotic Redundancy of the MTF Scheme for Stationary Ergodic Sources

The Move-to-front (MTF) scheme is a data-compression method which converts each symbol of a source sequence to a positive integer sequentially, and encodes it to a binary codeword. The compression performance of this algorithm has been analyzed usually under the assumption of the so-called symbol extension. But, in this paper, upper and lower bounds are derived for the redundancy of the MTF scheme without the symbol extension for stationary ergodic sources and Markov sources. It is also proved that for the stationary ergodic first-order Markov sources, the MTF scheme can attain the entropy rate if and only if the transition matrix of the source is a kind of doubly stochastic matrix. Moreover, if the source is a Kth-order Markov source (K/spl ges/2), the MTF scheme cannot attain the entropy rate of the source generally.

La dynamique des disparités régionales dans l'Union Européenne, 1980-1995

Résumé L’objectif de cet article est d’analyser la dynamique des disparités régionales dans l’Union Européenne sur la période 1980-1995 en se basant sur un échantillon de 138 régions. Premièrement, l’évolution de la forme de la distribution des PIB par tête est caractérisée par un test de a-convergence et par l’estimation non-paramétrique de fonctions de densité. Deuxièmement, le processus de croissance est modélisé à l’aide d’une chaîne de Markov de premier ordre et stationnaire. Troisièmement, l’influence de l’environnement géographique est analysée à l’aide d’une chaîne de Markov spatiale. Les résultats indiquent la persistance des disparités régionales, un biais progressif vers une trappe à pauvreté et l’importance de l’espace dans le processus de convergence.

Space-Time Analysis of GDP Disparities among European Regions: A Markov Chains Approach

The purpose of this article is to study the evolution of GDP disparities among 138 European regions over the period from 1980 to 1995. Regional per capitaGDPcross-sectional distribution is characterized by means of nonparametric estimations of density functions, and the growth process is modeled as a first-order stationary Markov chain. Spatial effects are then introduced within the Markov chain framework using regional conditioning and spatial Markov chains. The results of the analysis indicate the persistence of regional disparities, a progressive bias toward a poverty trap, and the importance of geography in the convergence process.

Evaluating the Markov Property in Studies of Economic Convergence

Markov chain theory, which has frequently been applied to analyze income convergence, imposes restrictive assumptions on the data-generating process. In most empirical studies, it is taken for granted that per capita income follows a stationary first-order Markov process. To examine the reliability of estimated Markov transition matrices, the authors propose Pearson X2 and likelihood ratio tests of the Markov property, spatial independence, and homogeneity over time and space. As an illustration, it is shown that per capita income in the forty-eight contiguous U.S. states did clearly not follow a common stationary first-order Markov process from 1929 to 2000.

Error bounds on multivariate Normal approximations for word count statistics

Given a sequence S and a collection Ω of d words, it is of interest in many applications to characterize the multivariate distribution of the vector of counts U = (N(S,w 1), …, N(S,w d )), where N(S,w) is the number of times a word w ∈ Ω appears in the sequence S. We obtain an explicit bound on the error made when approximating the multivariate distribution of U by the normal distribution, when the underlying sequence is i.i.d. or first-order stationary Markov over a finite alphabet. When the limiting covariance matrix of U is nonsingular, the error bounds decay at rate O ((log n) / √n) in the i.i.d. case and O ((log n)3 / √n) in the Markov case. In order for U to have a nondegenerate covariance matrix, it is necessary and sufficient that the counted word set Ω is not full, that is, that Ω is not the collection of all possible words of some length k over the given finite alphabet. To supply the bounds on the error, we use a version of Stein's method.

Error bounds on multivariate Normal approximations for word count statistics

Given a sequence S and a collection Ω of d words, it is of interest in many applications to characterize the multivariate distribution of the vector of counts U = (N(S,w1), …, N(S,wd)), where N(S,w) is the number of times a word w ∈ Ω appears in the sequence S. We obtain an explicit bound on the error made when approximating the multivariate distribution of U by the normal distribution, when the underlying sequence is i.i.d. or first-order stationary Markov over a finite alphabet. When the limiting covariance matrix of U is nonsingular, the error bounds decay at rate O((log n) / √n) in the i.i.d. case and O((log n)3 / √n) in the Markov case. In order for U to have a nondegenerate covariance matrix, it is necessary and sufficient that the counted word set Ω is not full, that is, that Ω is not the collection of all possible words of some length k over the given finite alphabet. To supply the bounds on the error, we use a version of Stein's method.