Dual Markov Research Articles

Dual Markov Chain and Dual Number Matrices with Nonnegative Standard Parts

Dual Markov Chain and Dual Number Matrices with Nonnegative Standard Parts

Flexible job shop scheduling via deep reinforcement learning with meta-path-based heterogeneous graph neural network

The flexible job shop scheduling problem (FJSP) is an important production scheduling problem in intelligent manufacturing. How to model the complex FJSP more accurately and improve the efficiency and generalization of scheduling policies is an urgent challenge to be solved. Therefore, a new end-to-end deep reinforcement learning (DRL) method combined with the meta-path-based heterogeneous graph neural network (MHGNN) is proposed to effectively solve FJSP. The task of solving FJSP is decomposed into two subtasks of operation selection and machine allocation. This dual-task FJSP is represented by introducing a heterogeneous graph and modeled as the dual Markov decision process (DMDP). A MHGNN is proposed to embed the global scheduling states of the dual-task FJSP. A heterogeneous graph neural network (GNN) framework is designed for the dual-task FJSP to efficiently encode the operation nodes and machine nodes, where the extracted embedded feature information is used to represent the operation selection policy and the machine allocation policy. Two policy networks are designed to efficiently predict the operation selection policy and the machine allocation policy, and a soft double-actors critic algorithm is proposed to train these two policy networks, where the trained policies can be used to efficiently solve FJSP instances of different scales. The experiments on three public benchmarks show that the proposed method outperforms the well-known heuristic scheduling rules and some advanced methods for solving FJSP. In particular, when solving large-scale FJSPs, the proposed method performs more prominently in terms of solution quality and solution time.

Asynchronous dissipative control for nonhomogeneous Markov jump systems with dual Markovian wireless fading channels

This article is devoted to the matter of asynchronous state feedback control for a class of nonhomogeneous Markov jump systems subjected to the dual Markovian wireless fading channel. A key feature here is that dual Markov chain is employed to model the stochastically switching of channel state with nonhomogeneous Markov process, which characterize both the transition of channel mode itself and the dependence on the system mode. Meanwhile, channel state-dependent packet dropout rate is time-varying with different channel mode. In the case of that, existing methods are difficult to deal with, resulting in the degradation of system performance. To this end, hidden Markov model-based asynchronous controller is firstly presented with compensated measurements. Secondly, co-designing with the asynchronous controller, a novel dynamic compensator is constructed to handle the mismatched measurements. Then, since both the Markov process of system and channel are considered, dual mode-dependent Lyapunov functional is constructed to verify the stochastic stability of the augmented system, as well as the strictly dissipative performance, which leads to less conservatism compared to the single mode-dependent one. Finally, a boost converter model driven by pulse-width-modulation is presented to demonstrate the validation of the proposed strategies.

A Measure-on-Graph-Valued Diffusion: A Particle System with Collisions and Its Applications

A diffusion-taking value in probability-measures on a graph with vertex set V, ∑i∈Vxiδi is studied. The masses on each vertex satisfy the stochastic differential equation of the form dxi=∑j∈N(i)xixjdBij on the simplex, where {Bij} are independent standard Brownian motions with skew symmetry, and N(i) is the neighbour of the vertex i. A dual Markov chain on integer partitions to the Markov semigroup associated with the diffusion is used to show that the support of an extremal stationary state of the adjoint semigroup is an independent set of the graph. We also investigate the diffusion with a linear drift, which gives a killing of the dual Markov chain on a finite integer lattice. The Markov chain is used to study the unique stationary state of the diffusion, which generalizes the Dirichlet distribution. Two applications of the diffusions are discussed: analysis of an algorithm to find an independent set of a graph, and a Bayesian graph selection based on computation of probability of a sample by using coupling from the past.

Integrating space syntax with spatial interaction

In this paper, we attempt to compare space syntax with spatial interaction. At one level, these two approaches to urban spatial structure are non-comparable. Space syntax is largely a descriptive technique for visualising spatial relations at the level of connections between places while spatial interaction is a predictive model that forecasts how much travel there will be between places. Space syntax articulates the system in terms of whether or not a physical link, usually at the level of the street, exists while spatial interaction predicts movements between all origins and destinations which are places often anchored in terms of the street network, but which at the level of prediction, assume connections between all places. Space syntax is grounded at a fine spatial scale while spatial interaction defines places as aggregates of activity in larger zones than the scale of the street system. The main output of space syntax is a connectivity matrix of step lengths between streets whereas in spatial interaction, such networks are predetermined, measurable in terms of Euclidean distance or generalised cost of travel, and the output is the volume of travel prior to this being assigned usually to a street network.There is however a fundamental way of relating the implicit network graph of spatial interaction to the explicit planar graph of the street network. We begin by assuming the planar graph of the network is conceived of as a primal problem of spatial interaction while the dual graph linking streets in the planar graph is the graph which is used in space syntax. We exploit this duality and show how we can move easily between spatial interaction as the primal and space syntax as the dual. This is rooted in a more fundamental graph – the bipartite graph which is a list of streets/arcs and their intersections/nodes from which the primal and dual emerge naturally. We explore various accessibility measures and show how they relate and correlate. We then go one step further and consider how various processes of random walking take place in these networks examining the steady states of the primal and dual problems in terms of the likelihood of a random walker visiting any node or street. We thus define primal and dual Markov chains that enable us to generate these probabilities. This provides a basic framework for comparing primal and dual in comparing spatial interaction with space syntax. We illustrate these measures on simple and easy to articulate graphs, extending this to a synthetic network of nearest neighbour links in Greater London based on 699 nodes and 1972 symmetric ‘streets’ between zones. This is a preliminary attack on the problem of linking these two approaches although many challenges remain.

Attack and Protection Technology of Intelligent Terminals for New Energy Internet Transmission, Transformation, and Distribution

In order to improve the ability of the new energy Internet to defend against external attacks, the author proposes a dual Markov chain-based FDIA detection method suitable for the new energy Internet. Taking into account the FDIA principles and characteristics of the new energy Internet, and the fact that the new energy Internet contains a large amount of measurement data and changing operating states, the data to be detected is mapped to two different state spaces, and two different Markovs are generated. The detector of FDIA is generated according to the accuracy of energy Internet operation state estimated by the model. The correctness and effectiveness of the proposed detection method are verified by experimental cases. Experimental results show: The proposed FDIA detection method has excellent detection probability and less detection calculation, the detection probability can reach 98.60%, and the false alarm probability is only 1.35%, compared with the support vector machine method, the calculation amount is reduced by an order of magnitude. It is proven that the method can meet the application requirements of the new energy Internet.

Few-shot activity learning by dual Markov logic networks

In Human activity recognition (HAR), a large amount of data may have no labels, so it is necessary to realize effective and credible data calibration under Few-shot learning (FSL). This paper proposes a new method for calibrating unlabeled data by using two models to improve the credibility of data labels further. Markov logic network (MLN) is used as the basic model. On the one hand, the construction of knowledge can reduce the dependence on the amount of data. On the other hand, the relationship between the actions can be effectively expressed. Specifically, we pre-train two MLN models using two unmatched training datasets. Then, these models are used to infer the possible labels of unlabeled data simultaneously. When labels are inconsistent, the most likely label will be selected and used to retrain the models. In order to ensure the credibility of the selected label, the selection process adopts a dual-model cross-validation method. This method uses the least square method to determine models’ weights according to the test results. Finally, it gives the possible probability of the labels under the common prediction of the models. Experimental results showed that the dual-model design method is better than the single model method in terms of time efficiency and better than the single model, TextCNN, TextLSTM, Transformer, and other models in terms of label credibility.

Duality for a class of continuous-time reversible Markov models

ABSTRACT Using a conditional probability structure we build transition probabilities that drive appealing classes of reversible Markov processes. The mechanism used in such a construction allows to find a dual Markov process. This kind of duality is then used to compute the predictor operator of one process via its dual. In particular, we identify the dual of some non-conjugate models, namely the queue model and a simple birth, death and immigration process. Such duals ensure that the computation of the predictor operators can be done via finite sums.

Intertwinings, second-order Brascamp–Lieb inequalities and spectral estimates

We explore the consequences of the so-called intertwinings between gradients and Markov diffusion operators on $R^d$ in terms of second-order Brascamp-Lieb inequalities for log-concave distributions and beyond, extending our inequalities established in a previous paper. As a result, we derive some convenient lower bounds on the $(d+1)^{th}$ positive eigenvalue depending on the spectral gap of the dual Markov diffusion operator given by the intertwining. To see the relevance of our approach, we apply our spectral results in the case of perturbed product measures, freeing us from Helffer's classical method based on uniform spectral estimates for the one-dimensional conditional distributions.

Hitting probabilities for the Greenwood model and relations to near constancy oscillation

We derive some properties of the Greenwood epidemic Galton–Watson branching model. Formulas for the probability $h(i,j)$ that the associated Markov chain $X$ hits state $j$ when started from state $i\ge j$ are obtained. For $j\ge1$, it follows that $h(i,j)$ slightly oscillates with varying $i$ and has infinitely many accumulation points. In particular, $h(i,j)$ does not converge as $i\to\infty$. It is shown that there exists a Markov chain $Y$ which is Siegmund dual to the chain $X$. The hitting probabilities of the dual Markov chain $Y$ are investigated.

Girsanov transformation for non-symmetric Markov processes

In this paper, we study the transformation of an arbitrary non-symmetric Markov process $X$ by multiplicative functionals which are the exponential of continuous additive functionals of $X$ with zero quadratic variations. We characterize the transformed semigroups by their associated quadratic forms. For a pair of dual Markov process $(X,\hat{X})$ associated with a non-symmetric Dirichlet form, we give a necessary and sufficient condition for the Girsanov transformed processes of $X$ and $\hat{X}$ to be dual with respect to another reference measure.

Convergence Rates and Limit Theorems for the Dual Markov Branching Process

This paper studies aspects of the Siegmund dual of the Markov branching process. The principal results are optimal convergence rates of its transition function and limit theorems in the case that it is not positive recurrent. Additional discussion is given about specifications of the Markov branching process and its dual. The dualising Markov branching processes need not be regular or even conservative.

Optimal Investment under Dual Risk Model and Markov Modulated Financial Market

In this paper, the optimal investment problem for an agent with dual risk model is studied. The financial market is assumed to be a diffusion process with the coefficients modulated by an external process, which is specified by the solution to a kind of stochastic differential equation. The object of the agent is to maximize the expected utility from terminal wealth. Together with the regularity property of the value function, by dynamic programming principle, the value function of our control problem is turned to be the unique solution to the associated Hamilton-Jacob-Bellman (HJB for short) equation. When the utility is an exponential function with constant risk aversion, close form expressions for value function and optimal investment policy are obtained.

Stochastic Duality of Markov Processes: A Study Via Generators

The article is devoted to a study of the duality of processes in the sense that for a certain f. This classical topic has well known applications in interacting particles, intertwining, superprocesses, stochastic monotonicity, exit – entrance laws, ruin probabilities in finances, etc. Aiming mostly at the case of f depending on the difference of its arguments, we develop a systematic approach to duality via the analysis of the generators of dual Markov processes and illustrate this approach by various examples, in particular, by giving a full characterization of duality arising from Pareto order in R d .

A Wiener-Hopf type factorization for the exponential functional of Lévy processes

For a Lévy process ξ=(ξt)t⩾0 drifting to −∞, we define the so-called exponential functional as follows: I ξ = ∫ 0 ∞ e ξ t d t . Under mild conditions on ξ, we show that the following factorization of exponential functionals: I ξ = d I H − × I Y holds, where × stands for the product of independent random variables, H− is the descending ladder height process of ξ and Y is a spectrally positive Lévy process with a negative mean constructed from its ascending ladder height process. As a by-product, we generate an integral or power series representation for the law of Iξ for a large class of Lévy processes with two-sided jumps and also derive some new distributional properties. The proof of our main result relies on a fine Markovian study of a class of generalized Ornstein–Uhlenbeck processes, which is itself of independent interest. We use and refine an alternative approach of studying the stationary measure of a Markov process which avoids some technicalities and difficulties that appear in the classical method of employing the generator of the dual Markov process.

A Symmetry Property for a Class of Random Walks in Stationary Random Environments on Z

A correspondence formula between the laws of dual Markov chains on Z with two transition jumps is established. This formula contributes to the study of random walks in stationary random environments. Counterexamples with more than two jumps are exhibited.

A Symmetry Property for a Class of Random Walks in Stationary Random Environments on Z

A correspondence formula between the laws of dual Markov chains on Z with two transition jumps is established. This formula contributes to the study of random walks in stationary random environments. Counterexamples with more than two jumps are exhibited.

A formula on scattering length of dual Markov processes

A formula on the scattering length for 3-dimensional Brownian motion was conjectured by M. Kac and proved by others later. It was recently proved under the framework of symmetric Markov processes by Takeda. In this paper, we shall prove that this formula holds for Markov processes under weak duality by the machinery developed mainly by Fitzsimmons and Getoor.

Exit Frequency Matrices for Finite Markov Chains

Consider a finite irreducible Markov chain on state spaceSwith transition matrixMand stationary distribution π. LetRbe the diagonal matrix of return times,Rii= 1/πi. Given distributions σ, τ andk∈S, the exit frequencyxk(σ, τ) denotes the expected number of times a random walk exits statekbefore an optimal stopping rule from σ to τ halts the walk. For a target distribution τ, we defineXτas then×nmatrix given by (Xτ)ij=xj(i, τ), whereialso denotes the singleton distribution on statei.The dual Markov chain with transition matrix=RM⊤R−1is called thereverse chain. We prove that Markov chain duality extends to matrices of exit frequencies. Specifically, for each target distribution τ, we associate a unique dual distribution τ*. Let$\rX_{\fc{\t}}$denote the matrix of exit frequencies from singletons to τ* on the reverse chain. We show that$\rX_{\fc{\t}} = R (X_{\t}^{\top} - \vb^{\top} \one)R^{-1}$, wherebis a non-negative constant vector (depending on τ). We explore this exit frequency duality and further illuminate the relationship between stopping rules on the original chain and reverse chain.

Duality in Interacting Particle Systems and Boson Representation

In the context of Markov processes, we show a new scheme to derive dual processes and a duality function based on a boson representation. This scheme is applicable to a case in which a generator is expressed by boson creation and annihilation operators. For some stochastic processes, duality relations have been known, which connect continuous time Markov processes with discrete state space and those with continuous state space. We clarify that using a generating function approach and the Doi-Peliti method, a birth-death process (or discrete random walk model) is naturally connected to a differential equation with continuous variables, which would be interpreted as a dual Markov process. The key point in the derivation is to use bosonic coherent states as a bra state, instead of a conventional projection state. As examples, we apply the scheme to a simple birth-coagulation process and a Brownian momentum process. The generator of the Brownian momentum process is written by elements of the SU(1,1) algebra, and using a boson realization of SU(1,1) we show that the same scheme is available.