One-step Transition Probability Matrix Research Articles

A Markov method for a mechanical system reliability assessment using discrete degradation data

Abstract A homogenous continuous-time Markov model (HCTMM) has the advantage of high accuracy and is usually used in reliability analysis. However, the transition intensities (rates) between any two states cannot be accurately obtained due to the discrete data obtained from a mechanical system and little expert knowledge. This paper proposes a method in which the transition intensity matrix of an HCTMM can be obtained indirectly using the actual discrete degradation data of a mechanical system. Firstly, the one-step transition probability matrix can be calculated by a hidden Markov model using the discrete degradation data. Secondly, the unit-time transition intensity matrix can be calculated from the one-step transition probability matrix. At last, an HCTMM is established to assess the reliability of a mechanical system based on the unit-time transition intensity matrix. Moreover, considering the maintenance activities cannot restore a mechanical system as a fire-new one, the random transition of a mechanical system after maintenance is expressed by a quasi-renewal process in this paper. Finally, three cases are studied to validate the proposed method.

The bilateral birth–death chain generated by the associated Jacobi polynomials

AbstractWe give a probabilistic interpretation of the associated Jacobi polynomials, which can be constructed from the three‐term recurrence relation for the classical Jacobi polynomials by shifting the integer index n by a real number t. Under certain restrictions, this will give rise to a doubly infinite tridiagonal stochastic matrix, which can be interpreted as the one‐step transition probability matrix of a discrete‐time bilateral birth–death chain with state space on . We also study the unique UL and LU stochastic factorizations of the transition probability matrix, as well as the discrete Darboux transformations and corresponding spectral matrices. Finally, we use all these results to provide an urn model on the integers for the associated Jacobi polynomials.

A Markov chain model for geographical accessibility

Accessibility analyses are conducted for a variety of applications, including urban planning and public health studies. These applications may aggregate data at the level of administrative units, such as provinces or municipalities. Accessibility between administrative units can be quantified by travel distance. However, modelling the distances between all administrative units in a region is computationally expensive if a large number of administrative units is considered. We propose a methodology to model accessibility between administrative units as a homogeneous Markov chain, where the administrative units are states and standardised inverse travel distances act as transition probabilities. Single transitions are allowed only between adjacent administrative units, resulting in a sparse one-step transition probability matrix (TPM). Powers of the TPM are taken to obtain transition probabilities between non-adjacent units. The methodology assumes that the Markov property holds for travel between units. We apply the methodology to administrative units within Tshwane, South Africa, considering only major roads for the sake of computation. The results are compared to those obtained using Euclidean distance, and we show that using network distance yields more reasonable results. The proposed methodology is computationally efficient and can be used to estimate accessibility between any set of administrative units connected by a road network.

Generalized Cell Mapping Method with Deep Learning for Global Analysis and Response Prediction of Dynamical Systems

A novel method that combines generalized cell mapping and deep learning is developed to analyze the global properties and predict the responses of dynamical systems. The proposed method only requires some prior knowledge of the system governing equations and obtains dynamical properties of the system from observed data. By combining the theoretical demonstration and empirical inference results, appropriate network structure and training hyperparameters are computed. Then a robust and efficient neural network approximation with the estimated mapping parameters is obtained. By using the approximate dynamical system model, we construct the one-step transition probability matrix and introduce the digraph analysis method to analyze the global properties. System responses at any time can be obtained with the trained model on the basis of the property of Markov chain. Several examples with periodic or chaotic attractors are presented to validate the proposed method. The influence of the number of hidden layers and the size of training data on calculated results is discussed, and an admissible architecture of the neural network is found. Numerical results indicate that the proposed method is quite effective for both global analysis and response prediction.

Analysis of the tendency of transition between segments of green consumer behavior with a Markov chain approach

PurposeThis study aims to determine the number of segments of green consumer behavior on toiletries products, and the tendency of transition between clusters is estimated. This study also provides recommendations based on the results.Design/methodology/approachThis study used primary data collected through an online and offline questionnaire. The questionnaire was intended to identify the socio-demographic characteristics, green consumer behavior state according to the environment as well as the willingness of the respondents to purchase various toiletries products (current, less green, and greener). Prior to segmenting green consumer behavior, scale purification using confirmatory factor analysis was performed to ensure the indicators used were valid. The k-means clustering algorithm was used for the segmentation, while discriminant analysis was used to validate the segmentation result. The Markov chain approach was performed to estimate the tendency of the transition between constructed segments, where the logistic regression model was applied to predict the individual transition probability.FindingsThe clustering algorithm resulted in three segments: light green, green and dark green. The light green segment has the lowest attitude toward the environmental criteria while the members of the dark green segment have the highest attitude among the other segments. The logistic regression indicated that the tendency of individuals to stay in the current segment or move to the adjacent segment was influenced by socio-demographic factors. The one-step transition probability matrix revealed that the tendency of a particular segment to move to the greener segment was greater than to stay or even move to the less green segment. The Markov chain approach then showed that the steady-state condition will emerge after 18 steps.Research limitations/implicationsThis study was limited geographically and by the criteria used for segmenting the green consumer behavior; therefore, it is recommended that this study be replicated on a greater scale with more criteria. A wider geographic area could be considered, including a national study, and more criteria, such as social influences, could be considered. This study does not focus on specific toiletries products. Selecting more specific toiletries products could be considered to provide a more reliable response from the respondents. Moreover, factors around the willingness to pay for green products were not investigated in greater detail although these factors might become indicators that can distinguish between two or more segments.Practical implicationsThis study empirically supports the theory that consumer environmentally friendly behavior can be used to appropriately categorize consumers into several segments, and thereby guide the development of a more differentiated policy approach for business and government.Social implicationsGreen consumer behavior may help save the environment and it will be beneficial in reducing environmental damage.Originality/valueThe study extends the existing literature related to green consumer behavior by segmenting the green consumer behavior based on the environmental criteria and applying the Markov chain approach to estimate the tendency of transition between segments.

Stochastic Response of a Vibro-Impact System via a New Impact-to-Impact Mapping

In this paper, a new impact-to-impact mapping is constructed to investigate the stochastic response of a nonautonomous vibro-impact system. The significant feature lies in the choice of Poincaré section, which consists of impact surface and codimensional time. Firstly, we construct a new impact-to-impact mapping to calculate the one-step transition probability matrix from a given impact to the next. Then, according to the matrix, we can investigate the stochastic responses of a nonautonomous vibro-impact system at the impact instants. The new impact-to-impact mapping is smooth and it effectively overcomes the nondifferentiability caused by the impact. A linear and a nonlinear nonautonomous vibro-impact systems are analyzed to verify the effectiveness of the strategy. The stochastic P-bifurcations induced by the noise intensity and system parameters are studied at the impact instants. Compared with Monte Carlo simulations, the new impact-to-impact strategy is accurate for nonautonomous vibro-impact systems with arbitrary restitution coefficients.

Towards the Saturation Throughput Disparity of Flows in Directional CSMA/CA Networks: An Analytical Model

Using directional antennas in wireless Ad hoc networks has many superiorities, including reducing interference, extending transmission range, and increasing space division multiplexing. However, directional transmission introduces two problems: deafness and directional hidden terminals problems. We observe that these problems result in saturation throughput disparity among the competing flows in directional CSMA/CA based Ad hoc networks and bring challenges for modeling the saturation throughput of the flows. In this article, we concentrate on how to model and analyze the saturation throughput disparity of different flows in directional CSMA/CA based Ad hoc networks. We first divide the collisions occurring in the transmission process into directional instantaneous collisions and directional persistent collisions. Then we propose a four-dimensional Markov chain to analyze the transmission state for a specific node. Our model has three different kinds of processes, namely back-off process, transmission process and freezing process. Each process contains a certain amount of continuous time slots which is defined as the basic time unit of the directional CSMA/CA protocols and the time length of each slot is fixed. We characterize the collision probabilities of the node by the one-step transition probability matrix in our Markov chain model. Accordingly, we can finally deduce the saturation throughput for each directional data stream and evaluate saturation throughput disparity for a given network topology. Finally, we verify the accuracy of our model by comparing the deviation of analytical results and simulation results.

Reduced rank multinomial logistic regression in Markov chains with application to cognitive data.

Finite Markov chains are useful tools for studying transitions among health states; these chains can be complex consisting of a mix of transient and absorbing states. The transition probabilities, which are often affected by covariates, can be difficult to estimate due to the presence of many covariates and/or a subset of transitions that are rarely observed. The purpose of this article is to show how to estimate the effect of a subset of covariates of interest after adjusting for the presence of multiple other covariates by applying multidimensional dimension reduction to the latter. The case in which transitions within each row of the one-step transition probability matrix are estimated by multinomial logistic regression is discussed in detail. Dimension reduction for the adjustment covariates involves estimating the effect of the covariates by a product of matrices iteratively; at each iteration one matrix in the product is fixed while the second is estimated using either standard software or nonlinear estimation, depending on which of the matrices in the product is fixed. The algorithm is illustrated by an application where the effect of at least one Apolipoprotein-E (APOE) gene allele on transition probability is estimated in a Markov Chain that includes adjustment for eight covariates and focuses on transitions from normal cognition to several forms of mild cognitive impairment, with possible absorption into dementia. Data were drawn from annual cognitive assessments of 649 participants enrolled in the BRAiNS cohort at the University of Kentucky's Alzheimer's Disease Research Center.

Research on stability modelling of high speed electronic communication based on hierarchical optimal mining algorithms

In order to solve the problems of low accuracy, bad network topological data integrity and low communication transmission efficiency in the current stability modelling methods for high-speed electronic communication, a new stability modelling method for high-speed electronic communication based on hierarchical optimisation mining algorithm is proposed. This paper analyses the transmission characteristics of high-speed signals in the process of electronic communication, constructs a wireless channel model by using hierarchical optimisation mining algorithm, and carries out distance correspondence based on the wireless channel model. The stability of high-speed electronic communication is modelled and analysed by using one-step transition probability matrix, node relative distribution and link duration. Experimental results show that this method has the advantages of high model accuracy, good topological data integrity and high communication transmission efficiency.

Research on stability modelling of high speed electronic communication based on hierarchical optimal mining algorithms

In order to solve the problems of low accuracy, bad network topological data integrity and low communication transmission efficiency in the current stability modelling methods for high-speed electronic communication, a new stability modelling method for high-speed electronic communication based on hierarchical optimisation mining algorithm is proposed. This paper analyses the transmission characteristics of high-speed signals in the process of electronic communication, constructs a wireless channel model by using hierarchical optimisation mining algorithm, and carries out distance correspondence based on the wireless channel model. The stability of high-speed electronic communication is modelled and analysed by using one-step transition probability matrix, node relative distribution and link duration. Experimental results show that this method has the advantages of high model accuracy, good topological data integrity and high communication transmission efficiency.

Computation of Invariant Measures and Stationary Expectations for Markov Chains with Block-Band Transition Matrix

This paper deals with the computation of invariant measures and stationary expectations for discrete-time Markov chains governed by a block-structured one-step transition probability matrix. The method generalizes in some respect Neuts’ matrix-geometric approach to vector-state Markov chains. The method reveals a strong relationship between Markov chains and matrix continued fractions which can provide valuable information for mastering the growing complexity of real-world applications of large-scale grid systems and multidimensional level-dependent Markov models. The results obtained are extended to continuous-time Markov chains.

On the Data-Driven Generalized Cell Mapping Method

Global analysis is often necessary for exploiting various applications or understanding the mechanisms of many dynamical phenomena in engineering practice where the underlying system model is too complex to analyze or even unavailable. Without a mathematical model, however, it is very difficult to apply cell mapping for global analysis. This paper for the first time proposes a data-driven generalized cell mapping to investigate the global properties of nonlinear systems from a sequence of measurement data, without prior knowledge of the underlying system. The proposed method includes the estimation of the state dimension of the system and time step for creating a mapping from the data. With the knowledge of the estimated state dimension and proper mapping time step, the one-step transition probability matrix can be computed from a statistical approach. The global properties of the underlying system can be uncovered with the one-step transition probability matrix. Three examples from applications are presented to illustrate a quality global analysis with the proposed data-driven generalized cell mapping method.

A Node Density Control Learning Method for the Internet of Things.

When examining density control learning methods for wireless sensor nodes, control time is often long and power consumption is usually very high. This paper proposes a node density control learning method for wireless sensor nodes and applies it to an environment based on Internet of Things architectures. Firstly, the characteristics of wireless sensors networks and the structure of mobile nodes are analyzed. Combined with the flexibility of wireless sensor networks and the degree of freedom of real-time processing and configuration of field programmable gate array (FPGA) data, a one-step transition probability matrix is introduced. In addition, the probability of arrival of signals between any pair of mobile nodes is also studied and calculated. Finally, the probability of signal connection between mobile nodes is close to 1, approximating the minimum node density at T. We simulate using a fully connected network identifying a worst-case test environment. Detailed experimental results show that our novel proposed method has shorter completion time and lower power consumption than previous attempts. We achieve high node density control as well at close to 90%.

Stochastic Darboux transformations for quasi-birth-and-death processes and urn models

We consider stochastic UL and LU block factorizations of the one-step transition probability matrix for a discrete-time quasi-birth-and-death process, namely a stochastic block tridiagonal matrix. The simpler case of random walks with only nearest neighbors transitions gives a unique LU factorization and a one-parameter family of factorizations in the UL case. The block structure considered here yields many more possible factorizations resulting in a much enlarged class of potential applications. By reversing the order of the factors (also known as a Darboux transformation) we get new families of quasi-birth-and-death processes where it is possible to identify the matrix-valued spectral measures in terms of a Geronimus (UL) or a Christoffel (LU) transformation of the original one. We apply our results to one example going with matrix-valued Jacobi polynomials arising in group representation theory. We also give urn models for some particular cases.

Modeling the Impact of Heterogeneous Transmission Powers on the Throughput of IEEE 802.11 Multi-Hop Ad Hoc Networks

Transmission power control (TPC) leads to heterogeneous transmission power levels of the nodes in multi-hop wireless ad hoc networks. This eventually results in a quite complicated collision scenario due to hidden and exposed terminal problems, and brings a big challenge for analyzing the saturation throughput of the flows. This paper presents a saturation throughput model for IEEE 802.11 multi-hop ad hoc networks with heterogeneous transmission powers. In the model, one tagged node goes through with three processes, i.e., backoff, freezing and transmission, and each of which is composed of a certain number of continuous fixed length time slots. We put forward a four-dimensional Markov chain model for the behavior of the tagged node in each fixed-length time slot. Under the condition of heterogeneous transmission powers, we consider two types of collisions, i.e., the instantaneous and persistent collisions. Their probabilities are characterized by the one-step transition probability matrix of the Markov chain. The per-flow saturation throughput of the flow is eventually derived through an iterative way. Finally, the accuracy of our model is validated by comparing the analytical values and the simulation results, and the impact of the heterogeneous power levels on the performance of the wireless multi-hop ad hoc networks is effectively analyzed.

Stochastic LU factorizations, Darboux transformations and urn models

Abstract We consider upper‒lower (UL) (and lower‒upper (LU)) factorizations of the one-step transition probability matrix of a random walk with the state space of nonnegative integers, with the condition that both upper and lower triangular matrices in the factorization are also stochastic matrices. We provide conditions on the free parameter of the UL factorization in terms of certain continued fractions such that this stochastic factorization is possible. By inverting the order of the factors (also known as a Darboux transformation) we obtain a new family of random walks where it is possible to state the spectral measures in terms of a Geronimus transformation. We repeat this for the LU factorization but without a free parameter. Finally, we apply our results in two examples; the random walk with constant transition probabilities, and the random walk generated by the Jacobi orthogonal polynomials. In both situations we obtain urn models associated with all the random walks in question.

Analysis of global properties for dynamical systems by a modified digraph cell mapping method

A modified procedure is proposed in this paper to solve the limitations of digraph cell mapping method in accurately analyzing the global properties of dynamical system, such as fractal basin boundaries. Firstly a rough cell structure is applied to the generalized cell mapping (GCM) method to obtain the general global properties of dynamical systems with digraph algorithm, and then a procedure based on the composite cell coordinate system method is proposed to increase the calculation accuracy. In order to further increase the calculation speed, a simple and feasible parallel strategy is applied during the creation process of one-step transition probability matrix for the GCM method. Meanwhile, the memory consumption can be greatly reduced by storing the one-step transition probability matrix as finite separate data files. The accurate global properties of two examples, a nonlinear oscillator with fractal basin structure and the Lorenz system, are demonstrated to show the effectiveness of our proposed improvement strategy.

The Stochastic Response of a Class of Impact Systems Calculated by a New Strategy Based on Generalized Cell Mapping Method

In this paper, a new strategy based on generalized cell mapping (GCM) method will be introduced to investigate the stochastic response of a class of impact systems. Significant difference of the proposed procedure lies in the choice of a novel impact-to-impact mapping, which is built to calculate the one-step transition probability matrix, and then, the probability density functions (PDFs) of the stochastic response can be obtained. The present strategy retains the characteristics of the impact systems, and is applicable to almost all types of impact systems indiscriminately. Further discussion proves that our strategy is reliable for different white noise excitations. Numerical simulations verify the efficiency and accuracy of the suggested strategy.

Optimal replacement policies for a shock model with a change point

Abstract In this paper, a mixed shock model with a change point is proposed to fit the situation that the failure mechanism changes during the system operation. For the new model, the change point appears when a run of damaging shocks with a prefixed length occurs, which divides the failure process of the system into two stages. Before the change point, the system is assumed to be failure-free due to the system’s own resistance to the shocks. After the change point, the system fails when the number of cumulative or consecutive damaging shocks reaches a corresponding critical value. A Markov chain and the one-step transition probability matrix are constructed based on the proposed model. Distributions of the time to the change point, the lifetime and the residual lifetime of the system are derived when the interarrival times between two consecutive shocks follow a general continuous phase-type distribution. For the new shock model, three replacement policies are designed to fit different monitoring situations of the system, and corresponding unconstrained optimization models are established to obtain the optimal replacement quantities. Numerical examples are presented to illustrate the proposed shock model and three replacement policies for the given form of the phase-type distribution. The proposed model can be applied to the actual railway track system and some new composite materials.

On the length and the position of the minimum sequence containing all runs of ones in a Markovian binary sequence

Let a sequence of binary (zero–one) trials which forms a nonhomogeneous/homogeneous Markov chain of first order with given initial probability distribution and one-step transition probability matrix. The trials are assumed to be ordered on a line. The statistics denoting the length and the position (starting, ending) of the shortest segment of the sequence containing all runs of ones, are considered. The study concerns with conditional probability distributions of the statistics given that there are at least two runs of ones in the sequence. Two application case studies related to learning experiments and DNA sequences are discussed. Illustrative numerics are presented.