Probabilistic Transition Research Articles

Coalgebraic Trace Semantics for Continuous Probabilistic Transition Systems

Coalgebras in a Kleisli category yield a generic definition of trace semantics for various types of labelled transition systems. In this paper we apply this generic theory to generative probabilistic transition systems, short PTS, with arbitrary (possibly uncountable) state spaces. We consider the sub-probability monad and the probability monad (Giry monad) on the category of measurable spaces and measurable functions. Our main contribution is that the existence of a final coalgebra in the Kleisli category of these monads is closely connected to the measure-theoretic extension theorem for sigma-finite pre-measures. In fact, we obtain a practical definition of the trace measure for both finite and infinite traces of PTS that subsumes a well-known result for discrete probabilistic transition systems. Finally we consider two example systems with uncountable state spaces and apply our theory to calculate their trace measures.

SLAP: Specification logic of actions with probability

A logic for specifying probabilistic transition systems is presented. Our perspective is that of agents performing actions. A procedure for deciding whether sentences in this logic are valid is provided. One of the main contributions of the paper is the formulation of the decision procedure: a tableau system which appeals to solving systems of linear equations. The tableau rules eliminate propositional connectives, then, for all open branches of the tableau tree, systems of linear equations are generated and checked for feasibility. Proofs of soundness, completeness and termination of the decision procedure are provided.

Łukasiewicz mu-Calculus

The paper explores properties of {\L}ukasiewicz mu-calculus, a version of the quantitative/probabilistic modal mu-calculus containing both weak and strong conjunctions and disjunctions from {\L}ukasiewicz (fuzzy) logic. We show that this logic encodes the well-known probabilistic temporal logic PCTL. And we give a model-checking algorithm for computing the rational denotational value of a formula at any state in a finite rational probabilistic nondeterministic transition system.

Discrete Time Stochastic Petri Box Calculus with Immediate Multiactions dtsiPBC

We propose discrete time stochastic Petri box calculus extended with immediate multiactions, called dt-siPBC. The step operational semantics is constructed via labeled probabilistic transition systems. The denotational semantics is defined via labeled discrete time stochastic Petri nets with immediate transitions (LDTSIPNs). A consistency of both semantics is demonstrated. In order to evaluate performance, the corresponding semi-Markov chains are analyzed. In a case study, performance of the shared memory system is evaluated.

Compositionality of Approximate Bisimulation for Probabilistic Systems

Probabilistic transition system specifications using the rule format ntmuft-ntmuxt provide structural operational semantics for Segala-type systems and guarantee that probabilistic bisimilarity is a congruence. Probabilistic bisimilarity is for many applications too sensitive to the exact probabilities of transitions. Approximate bisimulation provides a robust semantics that is stable with respect to implementation and measurement errors of probabilistic behavior. We provide a general method to quantify how much a process combinator expands the approximate bisimulation distance. As a direct application we derive an appropriate rule format that guarantees compositionality with respect to approximate bisimilarity. Moreover, we describe how specification formats for non-standard compositionality requirements may be derived.

Property value impacts of foreclosed housing acquisitions under uncertainty

Community development corporations seek to stabilize neighborhoods affected by the recent foreclosure crisis through acquisition and redevelopment of distressed properties. One rationale for this work is the alleviation or avoidance of negative foreclosure impacts. We estimate the lost value to proximate properties associated with a single foreclosure through a Markov chain representing probabilistic transitions between foreclosure stages. We apply our model to a case study of foreclosure properties in Chelsea, MA. A rank ordering by estimated property value impacts indicates significant potential gains in social value as compared to current community development practice. We extend our basic model to address the effects of clusters of foreclosed units upon the value of proximate properties. This study provides additional support for the use of decision modeling in foreclosed housing acquisition and redevelopment.

Probabilistic spatio-temporal retrieval in smart spaces

A ‘smart space’ is one that automatically identifies and tracks its occupants using unobtrusive biometric modalities such as face, gait, and voice in an unconstrained fashion. Information retrieval in a smart space is concerned with the location and movement of people over time. Towards this end, we abstract a smart space by a probabilistic state transition system in which each state records the probabilities of presence of individuals in various zones of the smart space. We carry out track-based reasoning on the states in order to determine more accurately the occupants of the smart space. This leads to a data model based upon an occupancy relation in which time is treated discretely, owing to the discrete nature of events, but probability is treated as a real-valued attribute. Using this data model, we show how to formulate a number of spatio-temporal queries, focusing on the computation of probabilities, an aspect that is novel to this model. We present queries both in SQL syntax and also in CLP(R), a constraint logic programming language (with reals) which facilitates succinct formulation of recursive queries. We show that the answers to certain queries are better displayed in a graphical manner, especially the movement tracks of occupants of the smart space. We also define query-dependent precision and recall metrics in order to quantify how well the model is able to answer various spatio-temporal queries. We show that a query-dependent metric gives significantly better results for a class of occupancy-related queries compared with query-independent metrics.

Fault detection and diagnosis in synchronous motors using hidden Markov model-based semi-nonparametric approach

Early detection and diagnosis of faults in industrial machines would reduce the maintenance cost and also increase the overall equipment effectiveness by increasing the availability of the machinery systems. In this paper, a semi-nonparametric approach based on hidden Markov model is introduced for fault detection and diagnosis in synchronous motors. In this approach, after training the hidden Markov model classifiers (parametric stage), two matrices named probabilistic transition frequency profile and average probabilistic emission are computed based on the hidden Markov models for each signature (nonparametric stage) using probabilistic inference. These matrices are later used in forming a similarity scoring function, which is the basis of the classification in this approach. Moreover, a preprocessing method, named squeezing and stretching is proposed which rectifies the difficulty of dealing with various operating speeds in the classification process. Finally, the experimental results are provided and compared. Further investigations are carried out, providing sensitivity analysis on the length of signatures, the number of hidden state values, as well as statistical performance evaluation and comparison with conventional hidden Markov model-based fault diagnosis approach. Results indicate that implementation of the proposed preprocessing, which unifies the signatures from various operating speeds, increases the classification accuracy by nearly 21% and moreover utilization of the proposed semi-nonparametric approach improves the accuracy further by nearly 6%.

THU0305 Cost-utility of tocilizumab in the treatment of systemic juvenile idiopathic arthritis in finland

BackgroundThe pathogenetic pathways and clinical presentation of systemic juvenile idiopathic arthritis (sJIA) differs from other forms of juvenile idiopathic arthritis (JIA). As a result the common JIA treatments are less...

Diffusion approximation for signaling stochastic networks

This paper introduces an unified approach to diffusion approximations of signaling networks. This is accomplished by the characterization of a broad class of networks that can be described by a set of quantities which suffer exchanges stochastically in time. We call this class stochastic Petri nets with probabilistic transitions, since it is described as a stochastic Petri net but allows a finite set of random outcomes for each transition. This extension permits effects on the network which are commonly interpreted as “routing” in queueing systems. The class is general enough to include, for instance, G-networks with negative customers and triggers as a particular case. With this class at hand, we derive a heavy traffic approximation, where the processes that drive the transitions are given by state-dependent Poisson-type processes and where the probabilities of the random outcomes are also state-dependent. The objective of this approach is to have a diffusion approximation which can be readily applied in several practical problems. We illustrate the use of the results with some numerical experiments.

Discrete ternary particle swarm optimization for area optimization of MPRM circuits

Having the advantage of simplicity, robustness and low computational costs, the particle swarm optimization (PSO) algorithm is a powerful evolutionary computation tool for synthesis and optimization of Reed-Muller logic based circuits. Exploring discrete PSO and probabilistic transition rules, the discrete ternary particle swarm optimization (DTPSO) is proposed for mixed polarity Reed-Muller (MPRM) circuits. According to the characteristics of mixed polarity OR/XNOR expression, a tabular technique is improved, and it is applied in the polarity conversion of MPRM functions. DTPSO is introduced to search the best polarity for an area of MPRM circuits by building parameter mapping relationships between particles and polarities. The computational results show that the proposed DTPSO outperforms the reported method using maxterm conversion starting from POS Boolean functions. The average saving in the number of terms is about 11.5%; the algorithm is quite efficient in terms of CPU time and achieves 12.2% improvement on average.

Analysis of the Debugging Model Based on Probabilistic State Transition

In general, software reliability test is an important task in software developing process. Meanwhile, it called perfect debugging which assumed that detecting a fault and fixing it, no introducing new faults, in turn, introducing new faults, called imperfect debugging. We all hope the testing process is perfect debugging and no new faults will be introduced. Therefore, it can shorten the testing time and decrease the testing costs. However, in practice, the software test is a complex, intractable process and can be divided into two categories, which are perfect debugging and imperfect debugging. Furthermore, most of testing processes are imperfect debugging. Then, what is relation between perfect debugging and imperfect debugging and what is impact to imperfect debugging transiting to perfect debugging? In this paper, we propose two state models based on probabilistic state transition to analyze the relation between perfect debugging and imperfect debugging and express debugging time and costs during imperfect debugging transiting to perfect debugging process in terms of the parameters in that model. Threshold conditions for perfect debugging process to be beneficial are also derived. Finally, using the numerical examples prove our assumptions and inference.

Learning Markov Decision Processes for Model Checking

Constructing an accurate system model for formal model verification can be both resource demanding and time-consuming. To alleviate this shortcoming, algorithms have been proposed for automatically learning system models based on observed system behaviors. In this paper we extend the algorithm on learning probabilistic automata to reactive systems, where the observed system behavior is in the form of alternating sequences of inputs and outputs. We propose an algorithm for automatically learning a deterministic labeled Markov decision process model from the observed behavior of a reactive system. The proposed learning algorithm is adapted from algorithms for learning deterministic probabilistic finite automata, and extended to include both probabilistic and nondeterministic transitions. The algorithm is empirically analyzed and evaluated by learning system models of slot machines. The evaluation is performed by analyzing the probabilistic linear temporal logic properties of the system as well as by analyzing the schedulers, in particular the optimal schedulers, induced by the learned models.

Tree rules in probabilistic transition system specifications with negative and quantitative premises

Probabilistic transition system specifications (PTSSs) in the ntmufnu/ntmuxnu format provide structural operational semantics for Segala-type systems that exhibit both probabilistic and nondeterministic behavior and guarantee that isimilarity is a congruence.Similar to the nondeterministic case of rule format tyft/tyxt, we show that the well-foundedness requirement is unnecessary in the probabilistic setting. To achieve this, we first define an extended version of the ntmufnu/ntmuxnu format in which quantitative premises and conclusions include nested convex combinations of distributions. This format also guarantees that bisimilarity is a congruence. Then, for a given (possibly non-well-founded) PTSS in the new format, we construct an equivalent well-founded transition system consisting of only rules of the simpler (well-founded) probabilistic ntree format. Furthermore, we develop a proof-theoretic notion for these PTSSs that coincides with the existing stratification-based meaning in case the PTSS is stratifiable. This continues the line of research lifting structural operational semantic results from the nondeterministic setting to systems with both probabilistic and nondeterministic behavior.

Probabilistic contracts for component-based design

We define a framework of probabilistic contracts for constructing component-based embedded systems, based on the formalism of discrete-time Interactive Markov Chains. A contract specifies the assumptions a component makes on its context and the guarantees it provides. Probabilistic transitions represent allowed uncertainty in the component behavior, for instance, to model internal choice or reliability. Action transitions are used to model non-deterministic behavior and communication between components. An interaction model specifies how components interact with each other. We provide the ingredients for a component-based design flow, including (1) contract satisfaction and refinement, (2) parallel composition of contracts over disjoint, interacting components, and (3) conjunction of contracts describing different requirements over the same component. Compositional design is enabled by congruence of refinement.

Determining expected behavior of fraudsters for a continuous audit system

This research attempts to determine the behavior of fraudsters in a continuous audit system where the fraudsters have multiple options for committing fraud. The system is modeled as a Continuous Time Markov Chain where the state changes are caused by the fraudster’s actions. The model uses a dynamic game with probabilistic transitions to determine the expected behavior of the fraudster. Managements and external auditors are expected to assess and evaluate controls that prevent and detect fraud. A key part of this assessment is determination of fraud-prone areas and the determination of those systems or sub-systems that are susceptible to fraud. There are several approaches that are used to find such areas. In this research, we attempt to model the behavior of fraudsters in order to help determine systems that require close monitoring. In this research we attempt to determine the behavior of fraudsters in a continuous audit system and where the fraudsters have multiple options for committing fraud. The system is modeled as a Continuous Time Markov Chain where the state changes are caused by the fraudster’s actions. In each state the fraudster can either choose to continue the fraud or to resign. The possibility of success in each case is assumed to be probabilistic. This state of affairs is represented as a dynamic game with probabilistic transitions. Since the fraudster assumes that audit system is a rational player who seeks to minimize the fraudster’s expected payoff, the Nash Equilibrium of the complete stochastic game is calculated. The Nash Equilibrium is a set of minimax vectors. These solution vectors represent a complete fraud strategy, which maximizes the expected payoff of the fraudster and ensures that she has no ex-post regrets.

Cost-effectiveness of the 21-gene recurrence score assay in the setting of multifactorial decision making for chemotherapy in early-stage breast cancer.

1525 Background: The objective of this study was to incorporate evidence from two recently-published studies to reevaluate the cost-effectiveness of the 21-gene Recurrence Score (RS) assay (Oncotype DX) in the context of multifactorial decision making to guide the use of chemotherapy for node-negative, estrogen receptor–positive breast cancer in the United States from the societal and healthcare system perspectives. Methods: We developed a decision-analytic model to first cross-classify hypothetical patients by clinicopathologic characteristics according to the Adjuvant! using risk groups and RS risk groups. We generated estimates of long-term costs, survival, and quality-adjusted survival for the RS-guided and non–RS-guided strategies using a probabilistic state transition model. In addition to costs for the 21-gene assay, we assigned attributable costs for chemotherapy, hormonal therapy, monitoring for disease recurrence, and distant recurrence. For the societal perspective, we also considered incremental patient time costs. Costs and survival were discounted at 3% annually. Results: With the RS-guided strategy, 40.4% of patients were expected to receive chemotherapy relative to 47.3% in the non–RS-guided strategy. Targeted use of chemotherapy in the RS-guided strategy was expected to increase survival by 0.19 years (95% CI, 0.09 to 0.32) and 0.16 QALYs (95% CI, 0.08 to 0.28). Lifetime direct medical costs were expected to be $2692 (95% CI, 1546 to 3821) higher with the RS-guided strategy. The incremental cost-effectiveness ratios (ICERs) were $14,059 per life-year saved (95% CI, $6840-$28,912) and $16,677 per QALY (95% CI, $7613-$37,219). When incorporating lower indirect costs of $950 per patient, the ICERs were $9095 per life-year saved (95% CI, dominant-$23,397) and $10,788 per QALY (95% CI, $6840-$30,265). In probabilistic sensitivity analysis, more than 99% of the ICERs were less than $50,000 per life-year saved and per QALY. Conclusions: Our updated cost-effectiveness estimates are supportive of the economic value of the 21-gene RS assay in the setting of node-negative, estrogen receptor–positive breast cancer.

Enhancing biometric recognition with spatio-temporal reasoning in smart environments

We discuss smart environments that identify and track their occupants using unobtrusive recognition modalities such as face, gait, and voice. In order to alleviate the inherent limitations of recognition, we propose spatio-temporal reasoning techniques based upon an analysis of the occupant tracks. The key idea underlying our approach is to determine the identity of a person based upon information from a track of events rather than a single event. We abstract a smart environment by a probabilistic state transition system in which each state records a set of individuals who are present in various zones of the smart environment. An event abstracts a recognition step, and the transition function defines the mapping between states upon the occurrence of an event. We express two forms of spatio-temporal reasoning in the form of transition functions: a track-based transition function and an error-correcting transition function. We also define the concepts of `precision' and `recall' to quantify the performance of the smart environment and provide experimental results to clarify the performance improvements from spatio-temporal reasoning. Our conclusion is that the state transition system is an effective abstraction of a smart environment and the application of spatial-temporal reasoning enhances the overall performance of a biometric recognition system.

Determining expected behaviour of fraudsters for a continuous audit system

This research attempts to determine the behaviour of fraudsters in a continuous audit system where the fraudsters have multiple options for committing fraud. The system is modelled as a Continuous Time Markov Chain where the state changes are caused by the fraudster's actions. The model uses a dynamic game with probabilistic transitions to determine the expected behaviour of the fraudster.

Bisimulation and simulation algorithms on probabilistic transition systems by abstract interpretation

We show how bisimulation equivalence and simulation preorder on probabilistic LTSs (PLTSs), namely the main behavioural relations on probabilistic nondeterministic processes, can be characterized by abstract interpretation. Both bisimulation and simulation can be obtained as completions of partitions and preorders, viewed as abstract domains, w.r.t. a pair of concrete functions that encode a PLTS. This approach provides a general framework for designing algorithms that compute bisimulation and simulation on PLTSs. Notably, (i) we show that the standard bisimulation algorithm by Baier et al. (2000) can be viewed as an instance of such a framework and (ii) we design a new efficient simulation algorithm that improves the state of the art.