Markov Renewal Model Research Articles

Applications of nature-inspired metaheuristic algorithms for tackling optimization problems across disciplines

Nature-inspired metaheuristic algorithms are important components of artificial intelligence, and are increasingly used across disciplines to tackle various types of challenging optimization problems. This paper demonstrates the usefulness of such algorithms for solving a variety of challenging optimization problems in statistics using a nature-inspired metaheuristic algorithm called competitive swarm optimizer with mutated agents (CSO-MA). This algorithm was proposed by one of the authors and its superior performance relative to many of its competitors had been demonstrated in earlier work and again in this paper. The main goal of this paper is to show a typical nature-inspired metaheuristic algorithmi, like CSO-MA, is efficient for tackling many different types of optimization problems in statistics. Our applications are new and include finding maximum likelihood estimates of parameters in a single cell generalized trend model to study pseudotime in bioinformatics, estimating parameters in the commonly used Rasch model in education research, finding M-estimates for a Cox regression in a Markov renewal model, performing matrix completion tasks to impute missing data for a two compartment model, and selecting variables optimally in an ecology problem in China. To further demonstrate the flexibility of metaheuristics, we also find an optimal design for a car refueling experiment in the auto industry using a logistic model with multiple interacting factors. In addition, we show that metaheuristics can sometimes outperform optimization algorithms commonly used in statistics.

Geometric Compound Hawkes Process and its Applications in Finance

The authors introduce a new model for a stock price, namely, geometric compound Hawkes process, and show how this model can be applied to solving many problems in finance, including European and American option pricing (perpetual American options), and Merton portfolio optimization problem. This model is a generalization of some well-known models in finance, such as Cox-Ross-Rubinstein model (1976) (geometric binomial process), Aase model (1988) (geometric compound Poisson process) and geometric Markov renewal model (2013). numerical examples are presented as well

High Frequency Trading and Asymptotics for Small Risk Aversion in a Markov Renewal Model

We study an optimal high frequency trading problem within a market microstructure model designed to be a good compromise between accuracy and tractability. The stock price is driven by a Markov renewal process (MRP), as described in [P. Fodra and H. Pham, Appl. Math. Finance, to appear], while market orders arrive in the limit order book via a point process correlated with the stock price itself. In this framework, we can reproduce the adverse selection risk, appearing in two different forms: the usual form due to big market orders impacting the stock price and penalizing the agent, and the weak form due to small market orders and reducing the probability of a profitable execution. We solve the market making problem by stochastic control techniques in this semi-Markov model. In the no risk-aversion case, we provide an explicit formula for the optimal controls and characterize the value function as a simple linear PDE. In the general case, we derive the optimal controls and the value function in terms of t...

A parametric Markov renewal model for predicting tropical cyclones in Bangladesh

In this paper, we consider a Markov renewal process (MRP) to model tropical cyclones occurred in Bangladesh during 1877–2009. The model takes into account both the occurrence history and some physical constraints to capture the main physical characteristics of the storm surge process. We assume that the sequence of cyclones constitutes a Markov chain, and sojourn times follow a Weibull distribution. The parameters of the Weibull MRP jointly with transition probabilities are estimated using the maximum likelihood method. The model shows a good fit with the real events, and probabilities of occurrence of different types of cyclones are calculated for various lengths of time interval using the model. Stationary probabilities and mean recurrence times are also calculated. A brief comparison with a Poisson model and a marked Poisson model has also been demonstrated.

High Frequency Trading in a Markov Renewal Model

We study an optimal high frequency trading problem within a market microstructure model aiming at a good compromise between accuracy and tractability. The stock price is modeled by a Markov Renewal Process (MRP), while market orders arrive in the limit order book via a point process correlated with the stock price, and taking into account the adverse selection risk. We apply stochastic control methods in this semi-Markov framework, and show how to reduce remarkably the complexity of the associated Hamilton-Jacobi-Bellman equation by suitable change of variables that exploits the specific symmetry of the problem. We then handle numerically the remaining part of the HJB equation, simplified into an integro-ordinary differential equation, by a bidimensional Euler scheme. Statistical procedures and numerical tests for computing the optimal limit order strategies illustrate our results.

Stayers in mixed Markov renewal models

In modelling the succession of states occupied by individuals over time, it is important to include state dependence, duration-of-stay effects, and variation between individuals over and above that explained by covariates. It has been long recognized that omission of any one of these characteristics can result in seriously misleading inference about the other two and the effects of covariates. Mixed Markov renewal models are the most parsimonious general class of model which incorporate all three characteristics. However, problems over the detailed specification and the complex computational requirements of such models have inhibited their use in social science research. This paper is concerned with one specific problem: ensuring sufficient flexibility for the multivariate mixing distribution by correcting the tendency to under predict the number who remain in the same state throughout. It is shown how an efficient EM-type algorithm based on weighted GLMs is readily extended to include stayers within a mixed Markov renewal formulation. In contrast to the direct maximization of the log-likelihood using a Newton-Raphson (or similar) algorithm, stayers may improve significantly the convergence performance of the EM-GLM approach.

Survival chance in papillary thyroid cancer in Hungary: individual survival probability estimation using the Markov method

Purpose: The typically benign, but occasionally rapidly fatal clinical course of papillary thyroid cancer has raised the need for individual survival probability estimation, to tailor the treatment strategy exclusively to a given patient. Materials and methods: A retrospective study was performed on 400 papillary thyroid cancer patients with a median follow-up time of 7.1 years to establish a clinical database for uni- and multivariate analysis of the prognostic factors related to survival (Kaplan-Meier product limit method and Cox regression). For a more precise prognosis estimation, the effect of the most important clinical events were then investigated on the basis of a Markov renewal model. The basic concept of this approach is that each patient has an individual disease course which (besides the initial clinical categories) is affected by special events, e.g. internal covariates (local/regional/distant relapses). On the supposition that these events and the cause-specific death are influenced by the same biological processes, the parameters of transient survival probability characterizing the speed of the course of the disease for each clinical event and their sequence were determined. The individual survival curves for each patient were calculated by using these parameters and the independent significant clinical variables selected from multivariate studies, summation of which resulted in a mean cause-specific survival function valid for the entire group. On the basis of this Markov model, prediction of the cause-specific survival probability is possible for extrastudy cases, if it is supposed that the clinical events occur within new patients in the same manner and with the similar probability as within the study population. Results: The patient's age, a distant metastasis at presentation, the extent of the surgical intervention, the primary tumor size and extent (pT), the external irradiation dosage and the degree of TSH suppression proved to be statistically significant and independent prognostic factors with regard to cause-specific survival in multivariate studies. During follow-up, 14, 14, 9 and 12% of the patients underwent local/ regional/distant relapses or thyroid cancer-related death, respectively. Through use of the above six independent clinical variables and the parameters relating to the four clinical events and their interrelations, mean cause-specific survival probabilities of 88, 83 and 78% were determined at 10, 20 and 30 years, respectively. The survival-predicting software (PATHYPRE) written on the basis of the biostatistical model is available through Internet connections on the home page of the National Institute of Oncology, Budapest (www.oncol.hu). Conclusion: Prediction of the individual survival probability for extrastudy cases affords a rationale for individualization of the treatment of papillary thyroid cancer patients.

Estimation of transition probabilities and bootstrap in a semiparametric markov renewal model

We consider parameter estimation based an iid sequence of censored observations of a finite state modulated renewal process. Its intensities assume a similar form as in Cox regression except that the baseline intensities are functions of the backwards recurrence time of the process while the covariates may depend on both calendar and duration time clock. Under conditions slightly stronger than in Andersen and gill we show that if the covariates depend on the backwards recurrence time only then the parameter estimates have the same asymptotic distribution as in the classical Cox regression. The choice of time independent covariates corresponds to Markov renewal processes. In this special case we discuss estimation of the renewal and transition probability matrices and show weak consistency of the bootstrap.

Cox Regression in a Markov Renewal Model: An Application to the Analysis of Bone Marrow Transplant Data

Abstract We discuss estimation and prediction in a Markov renewal model where intensities of the one-step transitions follow a Cox regression model with baseline hazards and covariates depending on the backward recurrence time. A modification of the Andersen and Gill results is given to extend the asymptotic properties of the partial likelihood estimates to this setting. We further consider prediction of the realization of the process in a single individual based on the follow-up information. Data of the International Bone Marrow Transplant Registry are used for illustration purposes.

Cox Regression in a Markov Renewal Model: An Application to the Analysis of Bone Marrow Transplant Data

Cox Regression in a Markov Renewal Model: An Application to the Analysis of Bone Marrow Transplant Data

A Markov-renewal extension of a deterministic state-variable model of the transport and fate of a toxic chemical in the environment

Specifications for stochastic Markov-renewal (MR) models are compared with those for deterministic, dynamic, state-variable (SV) models. Numerical predictions provided by MR models are qualified by probabilities. A case study is demonstrated in which the fate of the herbicide atrazine upon application in the watershed of an Iowa lake is tracked. Comparisons of numerical results on an MR model with those obtained in an earlier study employing an SV model show that explicit allowance for statistical variability in measured concentrations of a target chemical species assist substantially in interpreting differences between predicted and measured concentrations of the species. It is concluded that MR models provide a feasible alternative to SV models for predicting the fate of chemical species in aquatic environments in cases involving zero- or first-order kinetics of transfer and transformation of those species.

Correction to ‘A Markov renewal model for rainfall occurrences’ by E. Foufoula‐Georgiou and D. P. Lettenmaier

Correction to ‘A Markov renewal model for rainfall occurrences’ by E. Foufoula‐Georgiou and D. P. Lettenmaier

A Markov Renewal Model for rainfall occurrences

A probabilistic model for the temporal description of daily rainfall occurrences at a single location is presented. By defining an event as a day with measurable precipitation the model is cast into the discrete‐time point process framework. In the proposed model the sequence of times between events is formed by sampling from two geometric distributions, according to transition probabilities specified by a Markov chain. The model belongs to the class of Markov renewal processes and exhibits clustering relative to the independent Bernoulli process. As a special case, it reduces to a renewal model with a mixture distribution for the interarrival times. The rainfall occurrence model coupled with a mixed exponential distribution for the nonzero daily rainfall amounts was applied to the daily rainfall series for Snoqualmie Falls, Washington, and was successful in preserving the short‐term structure of the occurrence process, as well as the distributional properties of the seasonal rainfall amounts.

Markov Renewal Model for Maximum Bridge Loading

The prediction of maximum vehicle loadings on a bridge is studied. The stationary distribution of the static response of highway bridges under random truck loading is obtained using a Markov Renewal Model. This model is a generalization of Markov chains and renewal processes and can be used to model the arrival of trucks on a multilane bridge. The model also accounts for random truck characteristics such as axle weights, axle spacings, speed, and the headway distribution between trucks. The stationary distribution of the response is obtained assuming that the bridge (represented by its influence line) acts as a filter to the truck arrival process. The maximum lifetime response is obtained from the stationary distribution using an approximation to S. O. Rice's upcrossing rate formula. The results are then compared to a simulation program and acceptable agreement is reported.

The analysis of residence histories and other longitudinal panel data: A continuous time mixed markov renewal model incorporating exogeneous variables

The analysis of residence histories and other longitudinal panel data is fraught with methodological problems. Much recent progress has been made in methods of analysis within discrete time. This paper extends the development of empirically tractable mixed continuous time stochastic models. Analysis of a sample of intra-urban residential histories identifies the effect of tenure type, age of household head, size of household and duration of stay on movement probabilities. Surprisingly, no further variation, as represented by a gamma mixing distribution over a hazard rate parameter, may be identified.

Strongly consistent estimation in a controlled Markov renewal model

The optimal control of dynamic models which are not completely known to the controller often requires some kind of estimation of the unknown parameters. We present conditions under which a minimum contrast estimator will be strongly consistent independently of the control used. This kind of estimator is appropriate for the adaptive or ‘estimation and control' approach in dynamic programming under uncertainty. We consider a countable-state Markov renewal model and we impose bounding and recurrence conditions of the so-called Liapunov type.

Strongly consistent estimation in a controlled Markov renewal model

The optimal control of dynamic models which are not completely known to the controller often requires some kind of estimation of the unknown parameters. We present conditions under which a minimum contrast estimator will be strongly consistent independently of the control used. This kind of estimator is appropriate for the adaptive or ‘estimation and control' approach in dynamic programming under uncertainty. We consider a countable-state Markov renewal model and we impose bounding and recurrence conditions of the so-called Liapunov type.

Terminating markov renewal processes

Several kinds of terminating Markov Renewal Processes are defined. Of interest in these processes are the time T until termination and the number of transitions NT until termination. For several kinds of terminating processes, the distribution and moments of T and NT are obtained along with their covariance. The distributions of associated cumulative processes are also considered. A Markov Renewal model is compared with results of Markov Chains used to model epidemics, and other examples are examined in compartmental modeling and competing risks.

MAXIMUM LIKELIHOOD ESTIMATION OF PARAMETERS IN RENEWAL AND MARKOV‐RENEWAL PROCESSES

SummarySampling procedures using randomized observation‐points are suggested for estimating parameters in renewal and Markov renewal models. The usual asymptotic properties of the maximum likelihood method are shown to hold. The method we suggest provides a solution to the ML estimation problem in either or both of the following situations: (i) observations on between‐event intervals are unavailable, (ii) the interval densities are unknown or difficult to evaluate while their Laplace‐Stieltjes transforms are known.

A study of a Malthusian parameter in relation to some stochastic models of human reproduction

The paper is divided into six sections and is devoted to a study of a Malthusian parameter in relation to some stochastic models of human reproduction. In Section 1, some of the motivations underlying the study are discussed, and in Section 2 some literature on the stochastic model of population growth underlying the foundations of the paper is briefly reviewed. Section 3, which lays the foundations for the study of a more complicated model in Section 4, is devoted to the study of the Malthusian parameter in relation to a stochastic model of human reproduction formulated as a terminating renewal process. In Section 4 the Malthusian parameter is studied in relation to a terminating Markov renewal model of human reproduction, stemming from the work of Perrin and Sheps (1964). Among the mathematical results of independent interest in this section is a complete spectral decomposition of the Laplace-Stieltjes transform of the semi-Markov transition matrix in the model of Perrin and Sheps. Section 5 is devoted to the discussion of a mathematical method which allows accomodating in the model the time taken by an individual to reach reproductive age, and Section 6 ends the paper by supplying bounds for the Malthusian parameter which are valid under quite general conditions. Possible applications of the results in evaluating what influences a population policy may have on population growth are also discussed.