Markov Random Process Research Articles

Spectral method of analysis and optimal estimation in linear stochastic systems

It is proposed to use the spectral form of mathematical description of control systems for modeling continuous-time Markov random processes described by linear stochastic differential equations with additive or multiplicative noise. The obtained results are applied to solve the output process analysis problem and the optimal estimation problem.

Reliability Modeling for Multistate System with Preventive Maintenance under Customer Demand

The performance level of a multistate system (MSS) can vary among different values rather than only two states (perfect functioning and complete failure). To improve the reliability of MSSs, a maintenance strategy has been adopted to satisfy customer demand, and reliability modeling of MSS with preventive maintenance and customer demand is proposed. According to the regular degradation and random failure at each state, based on the Markov random process, the proposed MSS with preventive maintenance can be modeled to satisfy the customer demand in a specific state. This model can also be adapted to compute other reliability indices. Based on this model, the effect of different preventive maintenance actions on the reliability indices can be analyzed and further compared. Two numerical examples have been illustrated to show the validity of the proposed model. The reliability model presented in this study can be used to assess the type of MSS and help reliability engineers to compare different maintenance actions quantitatively and make optimal decisions.

Social Reliable D2D Relay for Trustworthy Paradigm in 5G Wireless Networks

5G wireless networks envision the usage of millimeter Wave (mmWave) frequencies for a manifold data rates and user capacity increase. However, the inherent line-of-sight (LoS) in mmWave communications raises significant challenges in providing seamless coverage across the dense 5G networks. Device-to-device (D2D) communications offer promising solutions for this LoS challenge by providing relay services. The selection of relay requires maintaining a certain level of trust and confidentiality of the devices. In this paper, we utilize social relationships among devices to form personal trust among different devices. Subsequently, we introduce distributed architecture for social relationships in D2D communications and we discuss new methods for a dynamic social relationship and trust management paradigm. Our analytical models not only show that the proposed scheme achieves 10% - 80% gain in device capacity, 30% - 200% increase in data rates, with approximately 50% less energy consumption over earlier methods, but also discuss co-existence with existing SINR based relay selection schemes. We present temporal dynamics of our social-communication graph using the random graph and Markov process. Our NS3 based 5G simulations demonstrate that our scheme obtains 3 - 5 times more throughput and 50 - 100m coverage extension, with relatively negligible additional delay.

Modifications of probabilistic models of states evolution for reliability analysis of district heating systems

The analysis and synthesis of reliability of energy systems, particularly district heating systems (DHS), is based on probabilistic modeling of different system states and events connecting these states (usually, these are failures and restoration of system components). The paper deals with the problem of modeling the probabilities of DHS states based on a Markov random process under conditions that can be characteristic for actual systems: joint and dependent failures of several components. For this purpose, it probabilistic models of the evolution of events in DHS taking into account non-ordinary events and dependent between them are developed. The methodology is based on the theory of Markov random processes and the basic laws of probability theory. The formation principles of developed models are illustrated by the state graphs. Numerical studies were carried out, the results of which allows determine some properties and application limits of developed models.

Search the optimal ratio of system’s component reliability parameters and the heat capacity of prosumers in the district heating system

The paper presents a methodology for ensuring the reliability of heat supply to prosumer connected to a district heating system, by using the heat capacity of its own heat source and optimization of failure rates of system components. To solve this problem, a mathematical formalization is developed, within which a markov random process, a flow distribution model of the theory of hydraulic circuits, nodal reliability indices, and simplified models of heat transfer processes in heat supply to consumers are used. A case study is considered based on a test scheme of district heating system with prosumer, an analysis of the results obtained is carried out, conclusions and directions for the further research on the subject are formulated.

Оценка вклада человеческого фактора в эксплуатационные характеристики сложных технических систем

Estimating the infl uence of human factor on the activity of operators of complex technical systems is an important problem for condition monitoring, personnel training and diagnostics. Presented are both an overview and mutual comparisons of the approaches which are useful to reveal the effect of human factor and have already shown their performances in practical applications. Under consideration are: the structural equation modeling, the Bayesian estimations for probabilistic models represented by Markov random processes, the multivariate statistical techniques including the discriminant and cluster analysis as well as wavelet transforms.

Статистические алгоритмы фильтрации для систем со случайно изменяющейся структурой

New algorithms for solving the optimal filtering problem for continuous-timesystems with a random structure are proposed. This problem is to estimate the currentsystem state vector from observations. The mathematical model of the dynamic systemincludes nonlinear stochastic differential equations, the right side of which defines the systemstructure (regime mode). The right side of these stochastic differential equations may bechanged at random time moments. The structure switching process is the Markov or conditionalMarkov random process with a finite set of states (structure numbers). The state vectorof such system consists of two components: the real vector (continuous part) and the integerstructure number (discrete part). The switch condition for the structure number may bedifferent: the achievement of a given surface by the continuous part of the state vector orthe distribution of a random time period between structure switchings. Each ordered pairof structure numbers can correspond to its own switch law. Algorithms for the estimation of the current state vector for systems with a randomstructure are particle filters, they are based on the statistical modeling method (MonteCarlo method). This work continues the authors’ research in the field of statistical methodsand algorithms for the continuous-time stochastic systems analysis and filtering.

Methodical approach to assessing the volume of youth labour migration in the states of the Eurasian Economic Union

Labour migration plays an integral role in the socio-economic development of any country; it affects both the structure and the size, the social and economic spheres of society. Therefore, it is necessary to monitor the migration process, because, in the future, it can have a positive impact on the economic growth of the country. The article proposes a model for macro-economic assessment of the volume of youth labor migration in the states of the Eurasian Economic Union (EAEU). The model is based on the ideas of labor migration in the form of a random Markov process with discrete states and continuous time. The chain states are interpreted as the labour market of the youth of the respective EAEU states, and the transition rates reflect the attractiveness of these markets for young people from different states. This made it possible to take into account the inherent uncertainty and inertia in the process of youth migration.

Optimization of the Effective Heat Supply Radius for the District Heating Systems

Abstract The problem of determining the effective (optimal) heat supply radius is considered. Heat supply radius is transportation distance of heat energy in the district heating systems (DHS), under which the highest indices of economic efficiency of district heating to consumers are respected. To solve this difficult and multifactorial problem, a bi-level approach has been proposed. This approach allows finding the optimal frameworks of territorial areas of district heating while fulfilling the necessary requirements for thermal-hydraulic modes in heat networks and for reliability of heating to consumers. Methodology for solving the formulated problem is based on bi-level programming methods, models of Theory of hydraulic circuits, nonlinear optimization methods, nodal reliability indices (availability factor, failure-free operation probability), Markov random processes models and other methods and models. A case study has been conducted using the developed methodological apparatus for the actual DHS scheme of the Irkutsk city (Russia, Siberia).

Diffusion Approximation for an Open Queueing Network with Limited Number of Customers and Time-Dependent Service

The object of this research is an open queueing network (QN) with a single class of customers, in which the total number of customers is limited. Service parameters are dependent on time, the routing of customers is determined by an arbitrary stochastic transition probability matrix, which is also depends on time. The service times of customers in each queue of the system is exponentially distributed with FIFO service, and it is assumed that a birth and death process generates and destroys the customers. The random vector, which determines the network state, forming a Markov random process is introduced. The purpose of the research is an asymptotic analysis of this Markov process, describing the queueing network state with a large number of customers, obtained from a system of differential equations, and used to find the mean relative number of customers in the network queues at any time. The results are illustrated with a specific example. This approach can be used tp model processes of customer service in the insurance companies, banks, logistics companies and other cyber-economical or service organizations.

Analysis of functioning of potato-terminal technological system based on probability-statistical approach

The process of harvesting potatoes with transport support is considered as the functioning of the potato-harvesting technological system consisting of the subsystems: person (P), machine (M), environment (E) and transport (T). The evaluation of the functioning of the system was carried out according to the developed methodology, based on a probabilistic-statistical approach, using the theory of Markov random processes and correlation and regression analysis. The applied method of time-keeping observations of the work of the potato harvester (tractor MTZ-80 + KPK-2-01) made it possible to carry out element-by-element analysis of the temporal characteristics of healthy and inoperable system states. According to the obtained statistics of the balance of the shift time, some regularity is established between the parameters of the system under consideration. The results of the research can be used to identify reserves for improving the functioning of potato technological systems, as well as to develop measures to reduce unproductive time spent in performing operational, technical, technological and organizational operations for harvesting potatoes.

VOS: A new outlier detection model using virtual graph

Outlier detection has been well studied due to its wide applications in both academia and industry, among which graph-based methods have drawn extensive attention in recent years because of their robust expressiveness for various types of datasets. Combining the local information with the implicit connections in the graph representation of the original dataset, in this study, we propose a new outlier detection model named Virtual Outlier Score (VOS). The proposed model constructs a similarity graph by using the top-k similar neighbors of each object, and introduces a virtual node coupling with a collection of virtual edges to generate a k-virtual graph. A tailored Markov random walk process is then performed on the strongly connected virtual graph under the principle that a potential outlier should get more weight to be visited. After reaching equilibrium, the stationary distribution vector is utilized to deduce the virtual outlier score. Furthermore, we provide a theoretical analysis of the proposed VOS model. Experiments on both synthetic and real-world datasets showed that the proposed model obtains an improvement over eight state-of-the-art algorithms under the measures of ROC AUC as well as the outlier discovery curve.

Matrices whose inverses are tridiagonal, band or block-tridiagonal and their relationship with the covariance matrices of a random Markov process

The article discusses the matrices of the form A1n, Amn, AmN, whose inverses are: tridiagonal matrix A-1n (n - dimension of the A-mn matrix), banded matrix A-mn (m is the half-width band of the matrix) or block-tridiagonal matrix A-m N (N = n x m - full dimension of the block matrix; m - the dimension of the blocks) and their relationships with the covariance matrices of measurements with ordinary (simple) Markov Random Processes (MRP), multiconnected MRP and vector MRP, respectively. Such covariance matrices frequently occur in the problems of optimal filtering, extrapolation and interpolation of MRP and Markov Random Fields (MRF). It is shown, that the structures of the matrices A1n, Amn, AmN have the same form, but the matrix elements in the first case are scalar quantities; in the second case matrix elements represent a product of vectors of dimension m; and in the third case, the off-diagonal elements are the product of matrices and vectors of dimension m. The properties of such matrices were investigated and a simple formulas of their inverses were found. Also computational efficiency in the storage and the inverse of such matrices have been considered. To illustrate the acquired results, an example on the covariance matrix inversions of two-dimensional MRP is given.

SYSTEM DYNAMICS MODEL OF TAX BEHAVIOR TAKING INTO ACCOUNT INCOME DISTRIBUTION

The problem of investigating the impact of taxation on the behavior of economic agents remains relevant within a long period of time. This is due, on the one hand, to the multiplicity of channels of this influence, which cover different aspects of people's activity (consumption, investment, employment etc.), and on the other – due to the importance of this issue for the economic government policy. One of the powerful channels of the influence of taxation on the economic agents’ activity of is opportunism as a behavioral property. The effect of this channel is tax evasion – one of the most pressing issues within this range of problems. The article proposes a system-dynamic model that takes into account traditional factors that influence tax evasion (tax control parameters), as well as the assumption that people's tax behavior can be determined by their financial status. The model allows to adjust the parameters of taxation and tax control, as well as the elasticity of reaction of economic agents as for tax evasion, depending on their level of income. Three groups of agents by income criterion are considered, the transitions between groups is modeled as a non-stationary Markov random process with the calculation of conditional transition probabilities at each time step, depending on the indicators of tax control effectiveness. Further directions of the study foresee detailed substantiation of the parameters of the model operators on the basis of the research of real regularities, that determine the intergroup dynamics and mechanisms of evasion, characteristic for different income groups. The model can be used as a virtual training ground, providing extensive experimentation options to test various theoretical assumptions and to study the influence of exogenous factors on the dynamics of endogenous variables, as well as building regression meta-models that will reflect the dependence of the selected response functions of the a number of exogenous factors, taking into account their combined effect.

Statistical approaches for the description formations and wear of the macrostructure of powder coatings and materials

Currently, powder metallurgy technologies are promising ways of hardening the surface of machine parts and mechanisms, obtaining wear-resistant materials for instrumental purposes. At the same time, the formation of the macrostructure of powder coatings and materials from a multitude of powder particles occurs under the continuous influence of numerous random factors, which practically excludes its dynamic description. Therefore, to describe the formation of the macrostructure of powder coatings and materials, the most promising are statistical approaches. The paper reviewed and analyzed literature data on the development and use of statistical approaches to describe the formation of the macrostructure of powder coatings and materials. As the review showed, statistical laws describing the macrostructure of wear resistant coatings and materials are also manifested in the process of their wear. The paper also presents statistical approaches developed by the authors based on the theory of Markov random processes and Monte Carlo methods. These statistical models are used to describe the formation of the macrostructure of powder coatings and materials and wear during sliding friction. The above developments reveal the statistical nature of collective phenomena occurring in powder metallurgy technologies. Models also establish uniform principles in describing the processes of formation and wear of the macrostructure of powder coatings and materials.

A novel segmented routing algorithm based on the markov decision process in mobile opportunistic networks

The purposes of research of routing algorithms in the mobile opportunistic networks are to improve the message delivery ratio, control overhead, and reduce the end‐to‐end delay. Now the performance of routing algorithms is very low because the opportunistic network structure is sparse and the network topology is variable. In this article, we use the Markov random process to propose a new segmented routing algorithm with a higher efficiency in opportunistic networks. In the stage of multicopy, we propose a forwarding strategy with the Markov decision process (MPD) to forward messages to multiple relay nodes quickly. In the stage of singe copy, we establish a precise delivery model to forward messages to the destination node with less relay nodes by combining the MPD with historical encountering information of nodes. The two‐stage segmented routing algorithm not only balances the relationship between the delivery ratio and the network overhead, but also saves network resources. Compared with other routing algorithms, our proposed segmented routing algorithm is superior to the existing routing algorithms in the message delivery ratio, message delay, and network overhead. © 2018 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Forecasting macro parameters representing the behavior of an applied multi-agent system

Under consideration is the method of probabilistic forecasting of macro parameters representing general patterns of an applied multi-agent system behavior. This system introduces the game interaction between a set of agents and a target. The behavior dynamics of such a system is described by a discrete-state discrete-time Markov random process and represented in the terms, which are considered convenient for interpretation and practical control of macro parameters. The corresponding calculations are based on common characteristics of the initial conditions. The probabilistic model of behavior of the system is generalized for the case of the mobile target and supplemented with formulas for the dynamical calculation of probabilistic distributions of the target’s defeat by the agents as well as the agent’s defeat by the target.

Nonlinear Approximation in the Large Deviations Principle

The Markov random processes and their approximations are considered. The main object of study is the exponential generator of random processes with independent increments, which are the solution of the problems of large deviations. These processes satisfy the conditions that make it possible to consider the Poisson and Levy approximation. Generators of random processes are normalized by nonlinear parameters. Found an explicit form of normalization parameter estimation.

Using Mathematical Models in Event Analysis from Military History

One of the ways of knowing the events of military history is to reproduce them using mathematical models. Based on the analysis of the fighting operations of the 4th Panzer Brigade of the Red Army in the vicinity of the city of Mtsensk in early October 1941, the capability to provide mathematical modeling of the fragments of these combat operations and the application of the apparatus of Markov random processes for these purposes is substantiated.The effectiveness of tanks depends not only on their technical properties, but also on the ways they are used on the battlefield. At the same time, combat effectiveness of tanks is commonly understood as their effectiveness in conditions when the methods of conducting combat operations by each of the opposing sides are the best.The battle outcome is probabilistic. It has certain regularity, depending on the combat tactics. The battle can be imagined as a multitude of randomly dueling fights between tanks, differing in their location and range of fire. A study of the probability of a system transition from each transient state to the next leads to the construction of mathematical models that allow calculating the ratio of losses of opposing sides.Based on the facts of military history and discovered regularities, the mathematical models are constructed to allow reproducing various fragments of combat according to the scheme of the Markov random process, and on their basis calculations are performed. The dependence of the ratio of the losses of the opposing sides depending on the number of firing positions used by the ambush tanks was established, provided that the change of these positions was made imperceptibly for the enemy.The obtained results can be used to develop tactical methods of using tanks in antiterrorist operations.

Virtual modelling of particles two-step feeding

The paper shows significant influence of mode and geometric parameters on the accuracy of particles feeding process. The model of the transformation of individual material batches into a continuous flow has been proposed based on the mathematical apparatus of random Markov processes with discrete space and time. The implementation and visualization of the mathematical model is carried out through the MAPLE universal mathematical package.