Markov Theory Research Articles

Using Markov Theory to Deliver Informed Decisions in Partially Observable Business Processes Operation

This article explores the stochastic capabilities offered by Markov theories combined with business transaction models, from the Enterprise Engineering field, to contribute to the decision-making body of knowledge. An agro-food case study shows the utility of this solution and the evaluation argues the management decisions value in situations where is not possible to fully observe the state of the reality, or to be fully aware about it. A full policy graph that forecasts the belief states from observations and enacted actions is delivered.

Concept of quantum timing jitter and non-Markovian limits in single-photon detection

Single atoms form a model system for understanding the limits of single photon detection. Here, we develop a non-Markov theory of single-photon absorption by a two-level atom to place limits on the absorption (transduction) time. We show the existence of a finite rise time in the probability of excitation of the atom during the absorption event which is infinitely fast in previous Markov theories. This rise time is governed by the bandwidth of the atom-field interaction spectrum and leads to a fundamental jitter in time-stamping the absorption event. Our theoretical framework captures both the weak and strong atom-field coupling regimes and sheds light on the spectral matching between the interaction bandwidth and single photon Fock state pulse spectrum. Our work opens questions whether such jitter in the absorption event can be observed in a multi-mode realistic single photon detector. Finally, we also shed light on the fundamental differences between linear and nonlinear detector outputs for single photon Fock state vs. coherent state pulses.

A STUDY OF THE PROBABILISTIC STATES OF MACHINETRACTOR AGGREGATES FOR WHEAT CULTIVATION

This paper applies Markov's theory of stochastic processes (Graph theory) to identify the probability of the machine-tractor aggregate being in each of the machine states adopted by the model. It features a graph of the states of a machine-tractor aggregate (Machine-status) operating in plant growing that allows us to determine the probability of the aggregate being in any of the states considered as basic. It proves that with the increase in the level of reliability of the aggregate, in this case its flawless performance, represented by the non-intermittent operation time indicator (time for trouble-free work), the probability that the aggregate will be in a functional state increases.

A Markov-Based Model for Information Security Risk Assessment in Healthcare MANETs

Information security breaches are of major concern for healthcare mobile ad-hoc networks (h-MANETs). In this paper, we propose a model that identifies and assesses risk in an h-MANET deployed by a healthcare institution in a disaster-prone region a priori by modeling the possible routes a hacker might follow to compromise a target. Our model proposes a novel method to compute the transition probability of each hop in the h-MANET. Next, it employs Markov theory to compute the maximum and minimum number of hops required to compromise the target for a given source-target pair. It then determines the vulnerability for all the paths comprising minimum to maximum hops only for each pair by computing their overall transition probability. Finally, our model computes the risk associated with each of these paths. Based on the calculated risk level of each path, the management can recommend an appropriate risk mitigation strategy.

Long-Term Extreme Response and Reliability of a Vessel Rolling in Random Beam Seas

In this paper, the long-term extreme response of a vessel rolling in random beam seas and the associated reliability evaluation are addressed. The long-term response analysis is based on the upcrossing rates of the roll motion under different sea states. Generally, for nonlinear roll motion in random seas, the high-level roll response is sensitive and closely related to the nonlinear effects associated with the restoring and damping terms. Therefore, assessing the corresponding statistics of the random roll motion with low probability levels is difficult and time-consuming. In this work, the Markov theory is introduced in order to tackle this problem. Specifically, for the dead ship condition, the random roll excitation moment is approximated as a filtered white noise process by applying a second-order linear filter and an efficient four-dimensional (4D) path integration (PI) technique is applied in order to calculate the response statistics. Furthermore, the reliability evaluation is based on the well-known Poisson estimate as well as on the upcrossing rate calculated by the 4D PI method. The long-term analysis and reliability evaluation of the nonlinear roll motion in random seas, which consider the variation of the sea states could be a valuable reference for ship stability research.

An early-stage design model for estimating ship evacuation patterns using the ship-centric Markov decision process

An early-stage design model is presented that estimates personnel locations on board a vessel during times of evacuation. This model takes into account various levels of uncertainty and pain that individuals may feel while heading toward safety, while simultaneously not requiring highly detailed information regarding the vessel layout. This makes this model suitable for analysis during early stages of design. To do this, principal eigenvector analysis is applied to the ship-centric Markov decision process model. Principal eigenvector analysis provides a leading indicator metric for forecasting and quantifying locations of individuals when coupled with the ship-centric Markov decision process model. For evacuation models suited for later stages of design, full temporal simulations may be required to understand long-term implications of personnel movement. This article proposes an alternative method that is able to identify some of these implications while not requiring full details of the vessel layout nor temporal simulations. To do this, a common theorem in Markov theory is applied that defines how the principal eigenvector represents the long-term steady-state behavior of the system. Metrics are defined that quantify the probability that an individual will congregate at specific locations on the vessels and highlight sensitivities to long-term behavior. A case study of a simplified vessel layout is presented that examines decision-making regarding ship egress analysis and general arrangements design. The results highlight specific areas of interest that cause significant changes to where individuals congregate and the probability they arrive safely at the exit. Sensitivity studies are performed varying the uncertainty in the movement of the individuals, how much pain they are experiencing, and one example where a passageway is blocked.

Investigation on Optimal Control of Hybrid Vehicle Energy Management System Based on Multi-source Traffic Information

Energy shortage and air pollution is an important problem facing the world. One of the effective ways to mitigate it is to develop hybrid vehicles and other energy resource vehicles. However, traffic decision information needed by hybrid vehicle energy management system has become the major bottleneck to restricting hybrid vehicle's popularization, which shows the characteristics of multi-source, time varying and fuzzy. Hybrid vehicle Energy management problem using multi- source traffic information cycle of s is taken as the aim, numerical simulations and laboratory tests are the two methods to solve it in this project. First, dynamic extraction mechanism of multi-source traffic information, such as dynamic link travel time and vehicle path are studied based on support vector regression theory and quantum ant colony algorithm. And then, Markov theory is adopted to study the system level energy optimization modeling of hybrid vehicle energy management system based on multi-source traffic information. Finally, using multi-source traffic information, quantitative control model of energy optimization performance is constructed .This topic is aim to reveal the energy optimal law in hybrid vehicle energy management system using multi-source traffic information, which would lay the theoretical and technical basis for developing new intelligent energy management system suitable for the conditions of multi-source traffic information.

Research on identification method of heavy vehicle rollover based on hidden Markov model

AbstractAiming at the problem of early warning credibility degradation as the heavy vehicle load and its center of gravity change greatly; the heavy vehicle rollover state identification method based on the Hidden Markov Model (HMM, is introduced to identify heavy vehicle lateral conditions dynamically in this paper. In this method, the lateral acceleration and roll angle are taken as the observation values of the model base. The Viterbi algorithm is used to predict the state sequence with the highest probability in the observed sequence, and the Markov prediction algorithm is adopted to calculate the state transition law and to predict the state of the vehicle in a certain period of time in the future. According to combination conditions of Double lane change and steering, applying Trucksim and Matlab trained hidden Markov model, the model is applied to the online identification of heavy vehicle rollover states. The identification results show that the model can accurately and efficiently identify the vehicle rollover state, and has good applicability. This study provides a novel method and a general strategy for active safety early warning and control of vehicles, which has reference significance for the application of the Hidden Markov theory in collision, rear-end and lane departure warning system.

Information Entropy in Predicting Location of Observation Points for Long Tunnel

Based on the Markov model and the basic theory of information entropy, this paper puts forward a new method for optimizing the location of observation points in order to obtain more information from limited geological investigation. According to the existing data from observation points data, classification of tunnel geological lithology was performed, and various lithology distribution were determined along the tunnel using the Markov model and theory. On the basis of the information entropy theory, the distribution of information entropy was obtained along the axis of the tunnel. Therefore, different information entropy could be acquired by calculating different classification of rocks. Furthermore, uncertainty increases when information entropy increased. The maximum entropy indicates maximum uncertainty and thus, this value determines the position of the new drilling hole. A new geology situation will be decided by the maximum entropy for the lowest accuracy. Optimal distribution will be obtained after recalculating, using the new location of the geology situation. Taking the engineering for the Bashiyi Daban water diversion tunneling in Xinjiang as a case, the maximum information entropy of the geological conditions was analyzed by the method proposed in the present study, with 25 newly added geology observation points along the axis of the 30-km tunnel. The results proved the validity of the present method. The method and results in this paper may be used not only to predict the geological conditions of underground engineering based on the investigated geological information, but also to optimize the distribution of the geology observation points.

TrackCC: A Practical Wireless Indoor Localization System Based on Less-Expensive Chips.

This paper aims at proposing a new wireless indoor localization system (ILS), called TrackCC, based on a commercial type of low-power system-on-chip (SoC), nRF24LE1. This type of chip has only l output power levels and acute fluctuation for a received minimum power level in operation, which give rise to many practical challenges for designing localization algorithms. In order to address these challenges, we exploit the Markov theory to construct a -sized state transition matrix to remove the fluctuation, and then propose a priority-based pattern matching algorithm to search for the most similar match in the signal map to estimate the real position of unknown nodes. The experimental results show that, compared to two existing wireless ILSs, LANDMARC and SAIL, which have meter level positioning accuracy, the proposed TrackCC can achieve the decimeter level accuracy on average in both line-of-sight (LOS) and non-line-of-sight (NLOS) senarios.

Optimal algorithms of data processing in navigation complexes of on-earth mobile objects with autonomous integrity monitoring of navigation data of satellite radio navigation systems

For navigation complexes of on-earth mobile objects, complex optimal algorithms for data processing in the vertical and horizontal planes are synthesized by methods of the Markov theory of estimation of random processes. On the basis of these algorithms, the structural scheme of a navigation complex is developed. The potential characteristics of the accuracy of the synthesized algorithms are calculated.

Target Lane Changing Prediction Method for ACC System

To improve the safety of adaptive cruise control system (ACC), experiment platform was built up. We obtained the movement state data of the host vehicle and vehicle ahead. Based on the distance between them, the lateral and longitudinal velocity, using Hidden Markov theory, we set up lane change intention prediction model of the vehicle ahead, which tested by the measured data. The results show that the prediction accuracy of the straight line section is 97% and in the curve section is 96%.

Stochastic phenotypic interconversion in tumors can generate heterogeneity.

Phenotype variations define heterogeneity in biological and molecular systems, and play a crucial mechanistic role, and heterogeneity has been demonstrated in tumor cells. In this work, cells from blood of patients affected by colon cancer were analyzed and sorted using a microfluidic assay based on galactose-active moieties and incubated for culturing in severe combined immunodeficiency (SCID) mice. Based on the results of these experiments, a model based on Markov theory is implemented and discussed to explain the equilibrium existing between phenotypes of cell subpopulations sorted using the microfluidic assay. In combination with the experimental results, the model has many implications for tumor heterogeneity; For example, it displays interconversion of phenotypes, confirming the experiments. Such interconversion generates metastatic cells and implies that targeting circulating tumor cells (CTC) will not be an efficient method for prevention of tumor recurrence. Most importantly, understanding the transitions between cell phenotypes in the cell population can improve understanding of tumor generation and growth.

Combined risk assessment of nonstationary monthly water quality based on Markov chain and time-varying copula.

Water quality risk management is a global hot research linkage with the sustainable water resource development. Ammonium nitrogen (NH3-N) and permanganate index (CODMn) as the focus indicators in Huai River Basin, are selected to reveal their joint transition laws based on Markov theory. The time-varying moments model with either time or land cover index as explanatory variables is applied to build the time-varying marginal distributions of water quality time series. Time-varying copula model, which takes the non-stationarity in the marginal distribution and/or the time variation in dependence structure between water quality series into consideration, is constructed to describe a bivariate frequency analysis for NH3-N and CODMn series at the same monitoring gauge. The larger first-order Markov joint transition probability indicates water quality state Class Vw, Class IV and Class III will occur easily in the water body of Bengbu Sluice. Both marginal distribution and copula models are nonstationary, and the explanatory variable time yields better performance than land cover index in describing the non-stationarities in the marginal distributions. In modelling the dependence structure changes, time-varying copula has a better fitting performance than the copula with the constant or the time-trend dependence parameter. The largest synchronous encounter risk probability of NH3-N and CODMn simultaneously reaching Class V is 50.61%, while the asynchronous encounter risk probability is largest when NH3-N and CODMn is inferior to class V and class IV water quality standards, respectively.

Sliding Mode Control for Discrete-Time Systems With Markovian Packet Dropouts

This paper presents the design of a sliding mode controller for networked control systems subject to successive Markovian packet dropouts. This paper adopts the Gilbert-Elliott channel model to describe the temporal correlation among packet losses, and proposes an update scheme to select the assumed available states for use in a sliding mode control law. A technique used in the theory of discrete-time Markov jump linear systems is applied to tackle the effect of the packet losses. This involves introducing a couple of Lyapunov functions dependent on the indicator functions of the instantaneous packet loss, and proving that the sliding mode controller is able to drive the system state trajectories into the neighborhood of the designed integral sliding surface in mean-square sense given that the corresponding Lyapunov inequalities are satisfied. The system is guaranteed thereafter to remain inside the neighborhood of the sliding surface. Simulated case studies are presented to illustrate the effectiveness of the control law.

Research on Markov property analysis of driving cycles and its application

The Markov property of driving cycles was discussed by making a thorough description of their essential characteristics. The Markov chain, a useful tool for designing and expressing driving cycles, has been increasingly used in the field of driving cycles in recent years. Although the Markov property of driving cycles left unproved, some researchers have taken it for granted that it is appropriate to design driving cycle with Markov chain. In our research, the vehicle dynamics model and the car-following model were used to establish the two-dimensional Markov state transition model. On the other hand, the driving data from the city of Changchun were collected to analyze how states (in Markov theory) correlation changes with the increase of time intervals. After the Markov property had been proven, the theory of ergodicity was applied to reveal the relationship between velocity-acceleration joint probability distribution (VA Probability) and state transition matrix. Finally, the application of Markov property was also discussed briefly. This research will lay a theoretical foundation for designing driving cycles and ECO driving (Economical and Ecological).

Markov-modulated continuous time finite Markov chain as the model of hybrid wireless communication channels operation

Problems of the development of hybrid communication systems based on laser and radio wave technologies are discussed. A novel method for assessing the reliability characteristics of such systems, operating in random environment, is presented. The method is based on the theory of Markov- modulated processes. An algorithm of stationary distribution calculation for the systems state probabilities is elaborated. Numerical examples illustrate the suggested approach.

Stochastic model of milk homogenization process using Markov's chain

The process of development of a mathematical model of the process of homogenization of dairy products is considered in the work. The theory of Markov's chains was used in the development of the mathematical model, Markov's chain with discrete states and continuous parameter for which the homogenisation pressure is taken, being the basis for the model structure. Machine realization of the model is implemented in the medium of structural modeling MathWorks Simulink™. Identification of the model parameters was carried out by minimizing the standard deviation calculated from the experimental data for each fraction of dairy products fat phase. As the set of experimental data processing results of the micrographic images of fat globules of whole milk samples distribution which were subjected to homogenization at different pressures were used. Pattern Search method was used as optimization method with the Latin Hypercube search algorithm from Global Optimization Тoolbox library. The accuracy of calculations averaged over all fractions of 0.88% (the relative share of units), the maximum relative error was 3.7% with the homogenization pressure of 30 MPa, which may be due to the very abrupt change in properties from the original milk in the particle size distribution at the beginning of the homogenization process and the lack of experimental data at homogenization pressures of below the specified value. The mathematical model proposed allows to calculate the profile of volume and mass distribution of the fat phase (fat globules) in the product, depending on the homogenization pressure and can be used in the laboratory and research of dairy products composition, as well as in the calculation, design and modeling of the process equipment of the dairy industry enterprises.

The probabilistic model of the process mixing of animal feed ingredients into a continuous mixer-reactor

A mathematical model of the polydisperse medium mixing process reflects its stochastic features in the form of uneven distribution of phase elements on the time of their presence in apparatus, particle size, ripple retention of the apparatus, random distribution of the material and thermal phase flows of the working volume, heterogeneity of the medium physical- and chemical properties, complicated by chemical reaction. For the mathematical description of the mixing process of animal feed ingredients in the presence of chemical reaction the system of differential equations of Academician V.V. Kafarov was used. Proposed by him hypothesis based on the theory of Markov’s processes stating that "any multicomponent mixture can be considered as the result of an iterative process of mixing the two components to achieve the desired uniformity of all the ingredients in the mixture" allows us to consider a process of mixing binary composition in a paddle mixer in the form of differential equations of two ingredients concentration numerous changes until it becomes a homogenous mixture. It was found out that the mixing process of the two-component mixture is determined in a paddle mixer with a constant mixing speed and a limit (equilibrium) dispersion of the ingredients in the mixture i.e. with its uniformity. Adjustment of the model parameters was carried out according to the results of experimental studies on mixing the crushed wheat with metallomagnetic impurity, which was a key (indicator) component. According to the best values of the constant of the continuous mixing speed and the equilibrium disperse values of the ingredients contents, the mathematical model parameters identification was carried out. The results obtained are used to develop a new generation mixer design.

Problem of Recognition of Hamiltonian Graph

In this article the author introduces the notions of combinatorial and of polynomial combinatorial sets in enumerative combinatorics. Formulates the problem of finding in combinatorial set of element with an easily recognizable property. The author proposes an efficient algorithm for solving this problem, which cancels known in the theory of algorithms abstract Turing, Church and Markov. We prove the criterion of polynomiality of the formulated problem. As a special case of this problem considers the problem of recognition of a Hamiltonian cycle in an undirected graph. We prove non-polynomiality this problem, which implies in particular the hypothesis of Jacques Edmonds P ≠NP.