Reduced State Space Research Articles

CONAIM: A Conscious Attention-Based Integrated Model for Human-Like Robots

Understanding consciousness is one of the most fascinating challenges of our time. From ancient civilizations to modern philosophers, questions have been asked on how one is conscious of his/her own existence and about the world that surrounds him/her. Although there is no precise definition for consciousness, there is an agreement that it is strongly related to human cognitive processes such as attention, a process capable of promoting a selection of a few stimuli from a huge amount of information that reaches us constantly. In order to bring the consciousness discussion to a computational scenario, this paper presents conscious attention-based integrated model (CONAIM), a formal model for machine consciousness based on an attentional schema for human-like agent cognition that integrates: short- and long-term memories, reasoning, planning, emotion, decision-making, learning, motivation, and volition. Experimental results in a mobile robotics domain show that the agent can attentively use motivation, volition, and memories to set its goals and learn new concepts and procedures based on exogenous and endogenous stimuli. By performing computation over an attentional space, the model also allowed the agent to learn over a much reduced state space. Further implementation under this model could potentially allow the agent to express sentience, self-awareness, self-consciousness, autonoetic consciousness, mineness, and perspectivalness.

Multi-core symbolic bisimulation minimisation

We introduce parallel symbolic algorithms for bisimulation minimisation, to combat the combinatorial state space explosion along three different paths. Bisimulation minimisation reduces a transition system to the smallest system with equivalent behaviour. We consider strong and branching bisimilarity for interactive Markov chains, which combine labelled transition systems and continuous-time Markov chains. Large state spaces can be represented concisely by symbolic techniques, based on binary decision diagrams. We present specialised BDD operations to compute the maximal bisimulation using signature-based partition refinement. We also study the symbolic representation of the quotient system and suggest an encoding based on representative states, rather than block numbers. Our implementation extends the parallel, shared memory, BDD library Sylvan, to obtain a significant speedup on multi-core machines. We propose the usage of partial signatures and of disjunctively partitioned transition relations, to increase the parallelisation opportunities. Also our new parallel data structure for block assignments increases scalability. We provide SigrefMC, a versatile tool that can be customised for bisimulation minimisation in various contexts. In particular, it supports models generated by the high-performance model checker LTSmin, providing access to specifications in multiple formalisms, including process algebra. The extensive experimental evaluation is based on various benchmarks from the literature. We demonstrate a speedup up to 95times for computing the maximal bisimulation on one processor. In addition, we find parallel speedups on a 48-core machine of another 17times for partition refinement and 24times for quotient computation. Our new encoding of the reduced state space leads to smaller BDD representations, with up to a 5162-fold reduction.

Stochastic model order reduction in randomly parametered linear dynamical systems

• Stochastic model order reduction for randomly parametered linear dynamical structures. • Reduction in state-space dimension using SEREP. • Development of surrogate model in the reduced state space using PCE. • Significant savings in computational costs for reliability analysis of large systems. • Proposed methodology enables using existing commercial FE softwares. This study focuses on the development of reduced order models for stochastic analysis of complex large ordered linear dynamical systems with parametric uncertainties, with an aim to reduce the computational costs without compromising on the accuracy of the solution. Here, a twin approach to model order reduction is adopted. A reduction in the state space dimension is first achieved through system equivalent reduction expansion process which involves linear transformations that couple the effects of state space truncation in conjunction with normal mode approximations. These developments are subsequently extended to the stochastic case by projecting the uncertain parameters into the Hilbert subspace and obtaining a solution of the random eigenvalue problem using polynomial chaos expansion. Reduction in the stochastic dimension is achieved by retaining only the dominant stochastic modes in the basis space. The proposed developments enable building surrogate models for complex large ordered stochastically parametered dynamical systems which lead to accurate predictions at significantly reduced computational costs.

Design and Optimisation of the FlyFast Front-end for Attribute-based Coordination

Collective Adaptive Systems (CAS) consist of a large number of interacting objects. The design of such systems requires scalable analysis tools and methods, which have necessarily to rely on some form of approximation of the system's actual behaviour. Promising techniques are those based on mean-field approximation. The FlyFast model-checker uses an on-the-fly algorithm for bounded PCTL model-checking of selected individual(s) in the context of very large populations whose global behaviour is approximated using deterministic limit mean-field techniques. Recently, a front-end for FlyFast has been proposed which provides a modelling language, PiFF in the sequel, for the Predicate-based Interaction for FlyFast. In this paper we present details of PiFF design and an approach to state-space reduction based on probabilistic bisimulation for inhomogeneous DTMCs.

Using plant model features to generate reduced test cases for programmable controllers

Complete conformance testing is a model-based test technique for programmable controllers. It checks whether an implementation conforms to its specifications with regard to all possible combinations of input signals, which is useful for small scale and safety critical systems. However, the state space explosion issue limits its application to large scale systems. This paper presents a method for reducing state space in generation of test cases by utilizing not only specification models but also features extracted from plant models. The application on a benchmark case study shows that the number of test cases is reduced significantly.

Estimation of Dynamic Models with Occasionally Binding Constraints.

We present an algorithm for estimating non-linear dynamic models, including those featuring occasionally binding constraints. The algorithm extends the Cubature Kalman Filter of Arasaratnam and Haykin (2009) with dynamic state space reduction, to give adequate speed in the presence of occasionally binding constraints, and to ensure that it can handle the large state spaces generated by pruned perturbation solutions to medium-scale DSGE models. We further extend the base algorithm to allow for alternative cubature procedures to improve the tracking of non-linearities. The algorithm relies on the solution method for models with occasionally binding constraints of Holden (2016b). We illustrate that the method can solve some of the identification problems that plague linearized DSGE models.

Controller synthesis for dynamic hierarchical real-time plants using timed automata

We use timed I/O automata based timed games to synthesize task-level reconfiguration services for cost-effective fault tolerance in a case study. The case study shows that state-space explosion is a severe problem for timed games. By applying suitable abstractions, we dramatically improve the scalability. However, timed I/O automata do not facilitate algorithmic abstraction generation techniques. The case study motivates the development of timed process automata to improve modeling and analysis for controller synthesis of time-critical plants which can be hierarchical and dynamic. The model offers two essential features for industrial systems: (i) compositional modeling with reusable designs for different contexts, and (ii) state-space reduction technique. Timed process automata model dynamic networks of continuous-time communicating plant processes which can activate other plant processes. We show how to establish safety and reachability properties of timed process automata by reduction to solving timed games. To mitigate the state-space explosion problem, an algorithmic state-space reduction technique using compositional reasoning and aggressive abstractions is also proposed. In this article, we demonstrate the theoretical framework of timed process automata and the effectiveness of the proposed state-space reduction technique by extending the case study.

Bayesian Analysis of Evolutionary Divergence with Genomic Data under Diverse Demographic Models

We present a new Bayesian method for estimating demographic and phylogenetic history using population genomic data. Several key innovations are introduced that allow the study of diverse models within an Isolation-with-Migration framework. The new method implements a 2-step analysis, with an initial Markov chain Monte Carlo (MCMC) phase that samples simple coalescent trees, followed by the calculation of the joint posterior density for the parameters of a demographic model. In step 1, the MCMC sampling phase, the method uses a reduced state space, consisting of coalescent trees without migration paths, and a simple importance sampling distribution without the demography of interest. Once obtained, a single sample of trees can be used in step 2 to calculate the joint posterior density for model parameters under multiple diverse demographic models, without having to repeat MCMC runs. Because migration paths are not included in the state space of the MCMC phase, but rather are handled by analytic integration in step 2 of the analysis, the method is scalable to a large number of loci with excellent MCMC mixing properties. With an implementation of the new method in the computer program MIST, we demonstrate the method's accuracy, scalability, and other advantages using simulated data and DNA sequences of two common chimpanzee subspecies: Pan troglodytes (P. t.) troglodytes and P. t. verus.

Fault trees on a diet: automated reduction by graph rewriting

Abstract Fault trees are a popular industrial technique for reliability modelling and analysis. Their extension with common reliability patterns, such as spare management, functional dependencies, and sequencing—known as dynamic fault trees (DFTs)—has an adverse effect on scalability, prohibiting the analysis of complex, industrial cases. This paper presents a novel, fully automated reduction technique for DFTs. The key idea is to interpret DFTs as directed graphs and exploit graph rewriting to simplify them. We present a collection of rewrite rules, address their correctness, and give a simple heuristic to determine the order of rewriting. Experiments on a large set of benchmarks show substantial DFT simplifications, yielding state space reductions and timing gains of up to two orders of magnitude.

Stick–Slip Motion of the Chaplygin Sleigh With a Piecewise-Smooth Nonholonomic Constraint

The Chaplygin sleigh is a canonical problem of mechanical systems with nonholonomic constraints. Such constraints often arise due to the role of a no-slip requirement imposed by friction. In the case of the Chaplygin sleigh, it is well known that its asymptotic motion is that of pure translation along a straight line. Any perturbations in angular velocity decay and result in an increase in asymptotic speed of the sleigh. Such motion of the sleigh is under the assumption that the magnitude of friction is as high as necessary to prevent slipping. We relax this assumption by setting a maximum value to the friction. The Chaplygin sleigh is then under a piecewise-smooth nonholonomic constraint and transitions between “slip” and “stick” modes. We investigate these transitions and the resulting nonsmooth dynamics of the system. We show that the reduced state space of the system can be partitioned into sets of distinct dynamics and that the stick–slip transitions can be explained in terms of transitions of the state of the system between these sets.

State space reduction techniques for model checking of MANET protocols

A Mobile Ad hoc Network (MANET) is a network that consists of mobile nodes and is autonomously managed without infrastructure base stations such as access points. MANETs have started being used as part of safety critical applications. A Vehicular Ad hoc Network (VANET) used in automated driving systems is such an example. In such applications, defects in the network protocol may cause serious social problems. Model checking, a state search-based verification technique, has proven to be effective in finding faults in complex system designs, such as communication protocols. However it is challenging to apply this technique to MANET protocols, because a MANET can have a number of different network topologies, thus resulting in the state explosion problem very easily. In this paper we propose a modeling technique to mitigate this problem using the AODV protocol as a running example. MANET protocols, such as AODV, typically enforce a source node that wishes to establish a route to the destination to retry the route establishing process some fixed number of times in face of failures. We show that to model check the protocol's behavior in these retries it suffices to consider only the last trial. The results of experiments using the SPIN model checker show that using the proposed technique significantly reduced the time and memory usage compared to standard full state exploration and allowed us to model check the protocol with up to five nodes.

Composed IIS path locating based optimization for bounded \\reachability analysis of compositional linear hybrid systems}{Composed IIS path locating based optimization for bounded reachability analysis of compositional linear hybrid systems

Hybrid systems include both discrete and continuous behavior and are widely used to model control systems. The reachability analysis of its unsafe state is an important method for guaranteeing the safety of a system. However, the current techniques do not scale well to the problems of practical interest. Due to the synchronization of components and the combinatorial explosion of state space, the reachability analysis of compositional linear hybrid system is extremely complex. In order to reduce the complexity, a path-oriented approach was proposed in a previous work, which conducted bounded reachability analysis of a compositional linear hybrid system. By enumerating and verifying each potential path one by one, the size of the problem that can be solved will be increased substantially. This path-oriented approach will become quite inefficient due to a sharp increase in the number of candidate paths when analyzing complex systems. The path explosion problem in model checking is also famous. To solve this problem, we propose a state-space reduction technique, which accelerates the verification process. We propose a method to locate the cause of infeasibility, when a composed infeasible path segment after a path set is proved to be infeasible. As we can simply falsify a path set that contains a composed infeasible path segment, the number of candidate paths can be reduced significantly. Furthermore, to avoid such composed path segments efficiently, we propose an approach based on satisfiability modulo theories (SMT), to traverse the bounded graph structure of the composed linear hybrid system. The results of the experiment show that the performance of the path-oriented bounded reachability analysis can be optimized significantly and that the overall performance of the proposed approach is better than that of the state-of-the-art competitor.

Adaptive Transit Routing in Stochastic Time-Dependent Networks

We define an adaptive routing problem in a stochastic time-dependent transit network in which transit arc travel times are discrete random variables with known probability distributions. We formulate it as a finite horizon Markov decision process. Routing strategies are conditioned on the arrival time of the traveler at intermediate nodes and real-time information on arrival times of buses at stops along their routes. The objective is to find a strategy that minimizes the expected travel time, subject to a constraint that guarantees that the destination is reached within a certain threshold. Although this framework proves to be advantageous over a priori routing, it inherits the curse of dimensionality, and state space reduction through preprocessing is achieved by solving variants of the time-dependent shortest path problem. Numerical results on a network representing a part of the Austin, Texas, transit system indicate a promising reduction in the state space size and improved tractability of the dynamic program.

State-space reduction and equivalence class sampling for a molecular self-assembly model

Direct simulation of a model with a large state space will generate enormous volumes of data, much of which is not relevant to the questions under study. In this paper, we consider a molecular self-assembly model as a typical example of a large state-space model, and present a method for selectively retrieving ‘target information’ from this model. This method partitions the state space into equivalence classes, as identified by an appropriate equivalence relation. The set of equivalence classes H, which serves as a reduced state space, contains none of the superfluous information of the original model. After construction and characterization of a Markov chain with state space H, the target information is efficiently retrieved via Markov chain Monte Carlo sampling. This approach represents a new breed of simulation techniques which are highly optimized for studying molecular self-assembly and, moreover, serves as a valuable guideline for analysis of other large state-space models.

Coded Retransmission in Wireless Networks Via Abstract MDPs: Theory and Algorithms

Consider a transmission scheme with a single transmitter and multiple receivers over a faulty broadcast channel. For each receiver, the transmitter has a unique infinite stream of packets, and its goal is to deliver them at the highest throughput possible. While such multiple-unicast models are unsolved in general, several network coding-based schemes were suggested. In such schemes, the transmitter can either send an uncoded packet, or a coded packet which is the function of a few packets. The packets sent can be received by the designated receiver (with some probability) or heard and stored by other receivers. Two functional modes are considered; the first presumes that the storage time is unlimited, while in the second one it is limited by the given time to expire (TTE) parameter. We model the transmission process as an infinite-horizon Markov decision process (MDP). Since the large state space renders exact solutions computationally impractical, we introduce policy-restricted and induced MDPs with significantly reduced state space, and prove that with proper reward function they have equal optimal value function (hence equal optimal throughput). We then derive a reinforcement learning algorithm, which learns the optimal policy for the induced MDP. This optimal strategy of the induced MDP, once applied to the policy of the restricted one, significantly improves over uncoded schemes. Next, we enhance the algorithm by means of analysis of the structural properties of the resulting cost functional. We demonstrate that our method scales well in the number of users, and automatically adapts to the packet loss rates, unknown in advance. In addition, the performance is compared to the recent bound by Wang, which assumes much stronger coding (e.g., intrasession and buffering of coded packets), yet is shown to be comparable.

Delay-Optimal Dynamic Mode Selection and Resource Allocation in Device-to-Device Communications—Part I: Optimal Policy

In this paper (Part I and Part II), we investigate the optimal dynamic mode selection and resource allocation to minimize the average end-to-end delay under a dropping probability constraint for an orthogonal frequency-division multiple-access (OFDMA) cellular network with device-to-device (D2D) communications. Different from the previous studies, which mostly focus on an infinite-backlog traffic model, we consider dynamic data arrival with nonsaturated buffers and formulate the resource control problem in D2D communications into an infinite-horizon average-reward constrained Markov decision process (CMDP) in Part I. The CMDP characterizes the dynamic interference between D2D links and cellular links based on their varying backlogged states, the dynamic route selection, and the coupled interactions between uplink and downlink resource allocations. We propose the general form of the optimal policy. In particular, it is proved that the optimal delay, respective of all feasible randomized policies, is attained by either a deterministic policy or a simple mixed policy, which randomizes between two deterministic policies. Therefore, the determination of an optimal randomized policy essentially becomes the determination of one or two deterministic policies, which can be obtained by an equivalent Bellman's equation with reduced state space. Simulation results show that the optimal policy based on the CMDP model outperforms the conventional channel-state-information-only scheme and the throughput-optimal scheme in stability sense.

Detecting intraday financial market states using temporal clustering

We propose the application of a high-speed maximum likelihood clustering algorithm to detect temporal financial market states, using correlation matrices estimated from intraday market microstructure features. We first determine the ex-ante intraday temporal cluster configurations to identify market states, and then study the identified temporal state features to extract state signature vectors (SSVs) which enable online state detection. The SSVs serve as low-dimensional state descriptors which can be used in learning algorithms for optimal planning in the high-frequency trading domain. We present a feasible scheme for real-time intraday state detection from streaming market data feeds. This study identifies an interesting hierarchy of system behaviour which motivates the need for timescale-specific state space reduction for participating agents.

Improved 3D Structure Prediction of Beta-Barrel Membrane Proteins by using Evolutionary Coupling Constraints, Reduced State Space and an Empirical Potential Function

Beta-barrel membrane proteins are found in the outer membrane of gram-negative bacteria, mitochondria, and chloroplasts. They carry out diverse biological functions, including pore formation, membrane anchoring, enzyme activity, and bacterial virulence. In addition, beta-barrel membrane proteins increasingly serve as scaffolds for bacterial surface display and nanopore-based DNA sequencing. Due to difficulties in experimental structure determination, they are sparsely represented in the protein structure databank and computational methods can help to understand their biophysical principles. We have developed a novel computational method to predict the 3D structure of beta-barrel membrane proteins using evolutionary coupling (EC) constraints and a reduced state space. Combined with an empirical potential function, we can successfully predict strand register at >80% accuracy for a set of 49 non-homologous proteins with known structures. This is a significant improvement from previous results using EC alone (44%) [1] and using empirical potential function alone (73%) [2]. Our method is general and can be applied to genome-wide structural prediction.[1] Sikander Hayat, Chris Sander, Debora S. Marks, and Arne Elofsson. All-atom 3D structure prediction of transmembrane β-barrel proteins from sequences. PNAS 2015 112 (17) 5413-5418.[2] Hammad Naveed, Yun Xu, Ronald Jackups, Jr., and Jie Liang. Predicting Three-Dimensional Structures of Transmembrane Domains of β-Barrel Membrane Proteins. Journal of the American Chemical Society 2012 134 (3), 1775-1781.

공간의존 파론도 게임의 재분배 모형

N명의 게임자들이 둥글게 둘러앉아 공간의존 파론도 게임 B를 실시한다. 게임 B는 여러 명의 게임자들 중에서 한 명을 임의로 선택하고, 선택된 게임자는 양 옆에 있는 두 명의 게임자들의 상태에 따라 앞면이 나올 확률이 달라지는 동전을 던져서 앞면이 나오면 1원을 얻고 뒷면이 나오면 1원을 잃는다. 게임 A'은 임의로 선택된 게임자가 나머지 N - 1명의 게임자들 중에서 한 명을 임의로 선택하여 본인의 상금 1원을 전달하는 게임으로 전체 게임자들의 총 상금에는 변함이 없으므로 전체 게임자들에게는 항상 공정한 게임이다. 만약 게임 B가 지는 게임인 반면에 두 게임 A'와 B를 결합한 혼합게임 C는 이기는 게임이 되면 파론도 효과가 존재하고, 게임 B가 이기는 게임이고 혼합게임 C는 지는 게임이면 역파론도 효과가 존재한다고 한다. 먼저 마코프 체인의 상태공간의 축소를 위한 lumpability 조건이 게임 A', B 그리고 혼합게임 C에 대해 만족함을 보이고, 축소된 상태공간에서 게임 B와 C의 기대상금을 계산한다. 이를 이용하여 파론도 효과와 역파론도 효과의 존재를 확인하고, 특별히 <TEX>$3{\leq}N{\leq}6$</TEX>의 경우에는 파론도 효과와 역파론도 효과가 존재하는 확률 모수의 영역을 도식화 한다. An ansemble of N players arranged in a circle play a spatially dependent Parrondo game B. One player is randomly selected to play game B, which is based on the toss of a biased coin, with the amount of the bias depending on states of the selected player's two nearest neighbors. The player wins one unit with heads and loses one unit with tails. In game A' the randomly chosen player transfers one unit of capital to another player who is randomly chosen among N - 1 players. Game A' is fair with respect to the ensemble's total profit. The games are said to exhibit the Parrondo effect if game B is losing and the random mixture game C is winning and the reverse-Parrondo effect if game B is winning and the random mixture game C is losing. We compute the exact mean profits for games B and C by applying a state space reduction method with lumped Markov chains and we sketch the Parrondo and reverse-Parrondo regions for <TEX>$3{\leq}N{\leq}6$</TEX>.

Co-optimizing Battery Storage for the Frequency Regulation and Energy Arbitrage Using Multi-Scale Dynamic Programming

We are interested in optimizing the use of battery storage for multiple applications, in particular energy arbitrage and frequency regulation. The nature of this problem requires the battery to make charging and discharging decisions at different time scales while accounting for the stochastic information such as load demand, electricity prices, and regulation signals. Solving the problem for even a single-day operation would be computationally intractable due to the large state space and the number of time steps. We propose a dynamic programming approach that takes advantage of the nested structure of the problem by solving smaller subproblems with reduced state spaces, over different time scales.