semi-Markov System Research Articles

Fully Connected Neural Network-Based Fixed-Time Adaptive Sliding Mode Control for Fuzzy Semi-Markov System

Fully Connected Neural Network-Based Fixed-Time Adaptive Sliding Mode Control for Fuzzy Semi-Markov System

Optimality criteria for the process of technical operation of locomotives

The efficiency of a locomotive fleet as a complex system depends significantly on the mode and conditions of its operation, which, in many cases, are determined both by its own condition and by the state of the maintenance system. In reality, changes in the state of the system, and its further behaviour in most cases can be described by the Markov process. It should be noted the excessive rigidity of the hypothesis about the Markov character of the behaviour in the process of technical operation of the locomotive fleet that makes the resulting model insufficiently adequate. The article proposes a methodology to assess the efficiency in the functioning of the locomotive fleet as a semi-Markov system. Relationships for calculating the main statistical characteristics of the efficiency indicators in the process of technical operation of locomotives have been obtained. The choice of optimality criteria for the process is justified. A procedure for constructing an ordered sequence of enhancing maintenance and repair strategies, considering the condition of a particular locomotive has been offered. Determining the optimality criteria for the technical operation process of the locomotive fleet based on the research of stochastic estimates in the efficiency of the semi-Markov model. Obtaining a sequence of maintenance and repair strategies. The efficiency of locomotive fleet operation significantly depends on the quality of service. The quality of service is determined by the effectiveness of diagnostic tools for technical conditions. The process of operating a locomotive fleet is described by a semi-Markov model. The semi-Markov model used makes it possible to more fully take into account, in the general case, the diagnostic support of the locomotive and the restoration process. The unit cost function the average unit cost, the average unit loss coefficient the coefficient of average, and operating factor was selected as target functions. A procedure has been proposed for constructing an ordered sequence of improving maintenance and repair strategies, considering the condition of a particular locomotive.

A Continuous-Time Semi-Markov System Governed by Stepwise Transitions

In this paper, we introduce a class of stochastic processes in continuous time, called step semi-Markov processes. The main idea comes from bringing an additional insight to a classical semi-Markov process: the transition between two states is accomplished through two or several steps. This is an extension of a previous work on discrete-time step semi-Markov processes. After defining the models and the main characteristics of interest, we derive the recursive evolution equations for two-step semi-Markov processes.

DEVELOPMENT OF A MODEL FOR THE DYNAMICS OF PROBABILITIES OF STATES OF SEMI-MARKOV SYSTEMS

The subject is the study of the dynamics of probability distribution of the states of the semi-Markov system during the transition process before establishing a stationary distribution. The goal is to develop a technology for finding analytical relationships that describe the dynamics of the probabilities of states of a semi-Markov system. The task is to develop a mathematical model that adequately describes the dynamics of the probabilities of the states of the system. The initial data for solving the problem is a matrix of conditional distribution laws of the random duration of the system's stay in each of its possible states before the transition to some other state. Method. The traditional method for analyzing semi-Markov systems is limited to obtaining a stationary distribution of the probabilities of its states, which does not solve the problem. A well-known approach to solving this problem is based on the formation and solution of a system of integral equations. However, in the general case, for arbitrary laws of distribution of the durations of the stay of the system in its possible states, this approach is not realizable. The desired result can only be obtained numerically, which does not satisfy the needs of practice. To obtain the required analytical relationships, the Erlang approximation of the original distribution laws is used. This technique significantly increases the adequacy of the resulting mathematical models of the functioning of the system, since it allows one to move away from overly obligatory exponential descriptions of the original distribution laws. The formal basis of the proposed method for constructing a model of the dynamics of state probabilities is the Kolmogorov system of differential equations for the desired probabilities. The solution of the system of equations is achieved using the Laplace transform, which is easily performed for Erlang distributions of arbitrary order. Results. Analytical relations are obtained that specify the desired distribution of the probabilities of the states of the system at any moment of time. The method is based on the approximation of the distribution laws for the durations of the stay of the system in each of its possible states by Erlang distributions of the proper order. A fundamental motivating factor for choosing distributions of this type for approximation is the ease of their use to obtain adequate models of the functioning of probabilistic systems. Conclusions. A solution is given to the problem of analyzing a semi-Markov system for a specific particular case, when the initial distribution laws for the duration of its sojourn in possible states are approximated by second-order Erlang distributions. Analytical relations are obtained for calculating the probability distribution at any time.

Phase Enlargement of Semi-Markov Systems without Determining Stationary Distribution of Embedded Markov Chain

A phase enlargement of semi-Markov systems that does not require determining stationary distribution of the embedded Markov chain is considered. Phase enlargement is an equivalent replacement of a semi-Markov system with a common phase state space by a system with a discrete state space. Finding the stationary distribution of an embedded Markov chain for a system with a continuous phase state space is one of the most time-consuming and not always solvable stage, since in some cases it leads to a solution of integral equations with kernels containing sum and difference of variables.
 For such equations there is only a particular solution and there are no general solutions to date. For this purpose a lemma on a type of a distribution function of the difference of two random variables, provided that the first variable is greater than the subtracted variable, is used.
 It is shown that the type of the distribution function of difference of two random variables under the indicated condition depends on one constant, which is determined by a numerical method of solving the equation presented in the lemma.
 Based on the lemma, a theorem on the difference of a random variable and a complicated recovery flow is built up. The use of this method is demonstrated by the example of modeling a technical system consisting of two series-connected process cells, provided that both cells cannot fail simultaneously. The distribution functions of the system residence times in enlarged states, as well as in a subset of working and non-working states, are determined. The simulation results are compared by the considered and classical method proposed by V. Korolyuk, showed the complete coincidence of the sought quantities.

Finding the probability distribution of states in the fuzzy markov systems

A problem on finding the stationary distributions of probabilities of states for the Markov systems under conditions of uncertainty is solved. It is assumed that parameters of the analyzed Markov and semi-Markov systems (matrix of transition intensities, analytical description of distribution functions of the durations of being in states of the system before exiting, as well as a matrix of transition probabilities) are not clearly assigned. In order to describe the fuzziness, we employ the Gaussian membership functions, as well as functions of the type. The appropriate procedure of systems analysis is based on the developed technology for solving the systems of linear algebraic equations with fuzzy coefficients. In the problem on analysis of a semi-Markov system, the estimation of components of the stationary distribution of probabilities of states of the system is obtained by the minimization of a complex criterion. The criterion considers the measure of deviation of the desired distribution from the modal one, as well as the level of compactness of membership functions of the fuzzy result of solution. In this case, we apply the rule introduced for the calculation of expected value of fuzzy numbers. The criterion proposed is modified through the introduction of weight coefficients, which consider possible differences in the levels of requirements to different components of the criterion.

Test method of distribution functions of the semi-markovian system dwell times in states

The article presents and proves the theorem on the stay times in the trajectory states taking the repeated returns into account. A method for determining the distribution functions of the stay times of the semi-Markov system in the states, called the method of trajectories, is introduced. An example of semi-Markovian modeling systems proving the correctness of the proposed method is examined.

Nonparametric estimation of interval reliability for discrete-time semi-Markov systems

In this article, we consider a repairable discrete-time semi-Markov system with finite state space. The measure of the interval reliability is given as the probability of the system being operational over a given finite-length time interval. A nonparametric estimator is proposed for the interval reliability, and the asymptotic properties of the strong consistency and the asymptotic normality for this estimator are proved. A numerical application concerning a four-state semi-Markov system is also presented.

Mission reliability of semi-Markov systems under generalized operational time requirements

Mission reliability of a system depends on specific criteria for mission success. To evaluate the mission reliability of some mission systems that do not need to work normally for the whole mission time, two types of mission reliability for such systems are studied. The first type corresponds to the mission requirement that the system must remain operational continuously for a minimum time within the given mission time interval, while the second corresponds to the mission requirement that the total operational time of the system within the mission time window must be greater than a given value. Based on Markov renewal properties, matrix integral equations are derived for semi-Markov systems. Numerical algorithms and a simulation procedure are provided for both types of mission reliability. Two examples are used for illustration purposes. One is a one-unit repairable Markov system, and the other is a cold standby semi-Markov system consisting of two components. By the proposed approaches, the mission reliability of systems with time redundancy can be more precisely estimated to avoid possible unnecessary redundancy of system resources.

On the Variances and Convariances of the Duration State Sizes of Semi-Markov Systems

In the present article we study the asymptotic behaviour of the sequence of vectors with components expectations, variances, and covariances of the state sizes of a semi-Markov system. In this respect, we transform the semi-Markov system into a Markov system with a different though equivalent state space and relate the sequence of the transition probabilities of the respective Markov system as functions of the parameters of the semi-Markov system. Also, we study the asymptotic behavior of the sequence of vectors with components variances and covariances of the duration state sizes, of the related Markov system, under perturbation of the transition probability matrices. We use the results in the study of asymptotic behavior under perturbation of the sequence of vectors with components variances and covariances of the semi-Markov system.

Markov Systems in a General State Space

In this article, we introduce and study Markov systems on general spaces (MSGS) as a first step of an entire theory on the subject. Also, all the concepts and basic results needed for this scope are given and analyzed. This could be thought of as an extension of the theory of a non homogeneous Markov system (NHMS) and that of a non homogeneous semi-Markov system on countable spaces, which has realized an interesting growth in the last thirty years. In addition, we study the asymptotic behaviour or ergodicity of Markov systems on general state spaces. The problem of asymptotic behaviour of Markov chains has been central for finite or countable spaces since the foundation of the subject. It has also been basic in the theory of NHMS and NHSMS. Two basic theorems are provided in answering the important problem of the asymptotic distribution of the population of the memberships of a Markov system that lives in the general space (X, ℬ(X)). Finally, we study the total variability from the invariant measure of the Markov system given that there exists an asymptotic behaviour. We prove a theorem which states that the total variation is finite. This problem is known also as the coupling problem.

Exotic Properties of Non Homogeneous Markov and Semi-Markov Systems

In this article we study what we chose to call exotic properties of NHMS and NHSMS. The interplay between stochastic theory of NHMS and NHSMS and other branches of probability, stochastic processes and mathematics, we believe is a fascinating one apart from being important. In many cases the information needed for the evolution of a NHMS is a larger set than the history of the multidimensional process NHMS. In our world where an overflow of information exists almost in all problems, it is almost surely that this will be available. Here, we extend the definition of the NHMS in order to accomodate this case. In this respect we arrive at the defnition of the 𝒢-non homogeneous Markov system. We study the problem of change of measure in a 𝒢-non homogeneousMarkov system. It is proved that under certain conditions the NHMS retains the Markov property, while as expected the basic sequences of transition probabilities change and it is established how they do so. We also find the expected population structure of the NHMS under the new measure in close analytic form. We also define the 𝒢-non homogeneous semi-Markov system and we study the problem of change of measure in a 𝒢-non homogeneous semi-Markov system. It is proved that under certain conditions the NHSMS retains the semi-Markov property while as expected the basic sequences of transition probabilities change and it is established how they do so. We prove that if the input process of memberships is a non homogeneous Poisson process, then asymtotically and under certain easily met in practice conditions, the compensated population structure of the 𝒢-NHMS is a martingale. Finally we prove that the space of all random population structures, under easily met in practice conditions, is a Hilbert space.

The Augmented Semi-Markov System in Continuous Time

This article proposes a continuous time semi-Markov hierarchical manpower planning model that incorporates the need of the employees to attend seminars, so as to enhance their prospects, as well as the organizations' intention to avoid situations concerning unavailability in skilled personnel when needed. At large, we study a hierarchical system where the workforce demand at each time period can be met via internal mobility and two streams of recruitment; one from the outside environment and another from a supplementary auxiliary system. For the suggested model, namely the Continuous Time Augmented Semi-Markov System, we examine initially its dynamic behavior by deriving the equations reflecting the expected number of persons in each grade. In the sequel, we probe its limiting population structure and it is found that under a set of conditions this structure exists and is specified. Finally, we present a real case which demonstrates the practical motivation of the subject under study.

The augmented semi-Markov system and its asymptotic behaviour

This paper suggests a generalized semi-Markov model for manpower planning, which could be adopted in cases of unavailability of candidates with the desired qualifications/experience, as well as in cases where an organization provides training opportunities to its personnel. In this context, we incorporate training classes into the framework of a non-homogeneous semi-Markov system and we introduce an additional, external semi-Markov system providing the former with potential recruits. For the model above, referred to as the Augmented Semi-Markov System, we derive the equations that reflect the expected number of persons in each grade and we also investigate its limiting population structure. An illustrative example is provided. Copyright © 2010 John Wiley & Sons, Ltd.

WTDM-Based ${\rm M}^{3}{\rm H}$ Filter for Target Tracking in the Presence of Outliers

This letter proposes a waiting-time-dependent semi-Markov (WTDM) switching based multiple model multiple hypothesis (M3H) filter to track a target in the presence of outliers. Two models, namely, a normal noise model and an outlier model, are constructed in the semi-Markov system. The adaptive transition probabilities are derived as the functions of the waiting time of outliers (or the interval of outlier occurrences), and this waiting time is treated as a discrete random variable governed by an exponential pmf. Performance of the WTDM-based M3H filter is demonstrated through simulation experiments. The proposed WTDM-based M3H filter outperforms the existing interacting multiple model (IMM) filter, which was proposed for the same purpose.

Stability of stochastic jump-parameter semi-Markov linear systems of differential equations

The asymptotic stability of stochastic Itô-type jump-parameter semi-Markov systems of linear differential equations is examined. A system of integral matrix equations is derived which has the property that the existence of a positive definite solution of the system implies the asymptotic stability of the stochastic semi-Markov system. Finally, an illustrative example is presented.

SYSTEMS RELIABILITY ANALYSIS IN VARIABLE OPERATION CONDITIONS

The semi-markov model of the system operation process is proposed and its selected parameters are defined. There are found reliability and risk characteristics of the multi-state series- "m out of k" system. Next, the joint model of the semi-markov system operation process and the considered multi-state system reliability and risk is constructed. The asymptotic approach to reliability and risk evaluation of this system in its operation process is proposed as well.

On the Optimal Coefficient of Efficiency of a Semi-Markov System in the Scheme of Phase Lumping

By using methods of the theory of semi-Markov processes, we analyze the problem of detecting signals in a multichannel system. We construct an optimal strategy for the motion of a search device in a multichannel system and obtain the corresponding estimate for the search efficiency.

Economic Rewards in Non-homogeneous Semi-Markov Systems

In this paper, a general reward model for a non homogeneous semi-Markov system will be provided. Both discrete and continuous time models will be studied considering reward to be a random variable of the economic type associated with the state occupancies and transitions. Basic equations will be exhibited and the main formulas for the expected reward that the system will generate in some interval of time (with and without discounting) will be determined for the discrete and continuous cases and a method is provided in order to find it in closed analytic form in relation with the basic parameters of the system. Also, results will be obtained for the expected reward per time period for the steady state.

Discrete Time Reward Models for Homogeneous Semi-Markov Systems

Previously, continuous time Markov and semi-Markow models have been developed for a multi-grade system with a number of transient states and a single absorbing state. Such systems have been studied with Poisson arrivals with known growth. Such models have been applied to manpower planning and the movement of patients through a hospital system. In this paper, we develop a reward model for a discrete time homogeneous semi-Markov system with Poisson arrivals and the same model where the system has growth with known size at each time point. Discrete time is employed here to accommodate a discrete reward structure. Results are obtained for the distribution, and mean of daily rewards of such system at any time, and in steady state; the expected reward in a period of time is also determined. These results are illustrated using data on employees moving through a company hierarchy and data for hospital patients.