Mixture Expectation Maximization Algorithm Research Articles

Censored expectation maximization algorithm for mixtures: Application to intertrade waiting times

In a previous analysis the problem induced by “zero-inflation” in time series data (caused by high frequency trading in the electronic order book) was handled by left-truncating the waiting times between consecutive limit orders . We demonstrated, using rigorous statistical methods, that the truncated Weibull distribution describes the corresponding stochastic dynamics for the entire range of inter-arrival limit order waiting times, except for a region close to zero. However, since the truncated Weibull distribution was not able to describe the prodigious “zero-inflated” probability mass in the neighbourhood of zero (making up approximately 50% of the data for limit orders), it became clear that the entire probability distribution must be a mixture distribution of which the Weibull distribution is a significant part. To investigate this idea, we use a “censored expectation–maximization algorithm” to analyse the intertrade waiting times data for four selected stocks trading on the London Stock Exchange. The intertrade waiting times usually have a much lower percentage of zero inflation, typically around 2.5%. Making use of this new method and testing various mixture models, we show that the desired mixture consists of the Weibull distribution with the universal shape parameter of β≃0.57 plus an additional exponential distribution. This is the same value for the shape parameter found already in our previous study. The “1 exponential + 1 Weibull” mixture describes the intertrade waiting times extremely well at all time scales. While the Weibull component dominates in the transition and tail regions the exponential distribution explains the “zero-inflated” excess mass.

Asymptotic Convergence Properties of the EM Algorithm for Mixture of Experts

Mixture of experts (ME) is a modular neural network architecture for supervised classification. The double-loop expectation-maximization (EM) algorithm has been developed for learning the parameters of the ME architecture, and the iteratively reweighted least squares (IRLS) algorithm and the Newton-Raphson algorithm are two popular schemes for learning the parameters in the inner loop or gating network. In this letter, we investigate asymptotic convergence properties of the EM algorithm for ME using either the IRLS or Newton-Raphson approach. With the help of an overlap measure for the ME model, we obtain an upper bound of the asymptotic convergence rate of the EM algorithm in each case. Moreover, we find that for the Newton approach as a specific Newton-Raphson approach to learning the parameters in the inner loop, the upper bound of asymptotic convergence rate of the EM algorithm locally around the true solution Θ* is [Formula: see text], where ϵ>0 is an arbitrarily small number, o(x) means that it is a higher-order infinitesimal as x → 0, and e(Θ*) is a measure of the average overlap of the ME model. That is, as the average overlap of the true ME model with large sample tends to zero, the EM algorithm with the Newton approach to learning the parameters in the inner loop tends to be asymptotically superlinear. Finally, we substantiate our theoretical results by simulation experiments.