Stochastic Action Research Articles

Performance evaluation from stochastic statecharts representation of flexible reactive systems: A simulation approach

This paper focuses on the performance analysis of flexible reactive systems. The performance analysis consists of two phases: first system modeling, second performance evaluation. The paper models the flexible reactive system by the stochastic statecharts method, and uses the simulation method to evaluate the performance. To make use of the feature of event-triggered state transitions in the statecharts, a new method of simulation is proposed based on the techniques of the discrete-event system simulation. The new method solves the problem of computer implementation of stochastic events, probabilistic transition, concurrent states, parallel actions, and broadcast communication mechanism in the stochastic statecharts. An example of a flexible manufacturing system is presented. The simulation result of the example is consistent with the analytical result, which shows the feasibility of the proposed new simulation method.

Analysis of non-linear stochastic oscillations by the averaging method

The solution of a quasiconservative non-linear oscillatory system is considered, the right-hand sides of which are proportional to a small parameter. Fundamental relations for solving the problem are obtained by changing to “slow” variables and a combination of the stochastic averaging method and the theory of Markov processes. An efficient numerical algorithm is developed based on the fast Fourier transform that enables the output parameter distribution density and the amplitudes of the oscillations to be obtained. Application of the theory to solve the Duffing–van der Pol equation for an additive and multiplicative stochastic action is considered.

Reliability of the Offshore Jacket Platform Based on Load Combination

Offshore jacket platforms under complicated and changeful ocean environments are subjected to various stochastic actions such as actions of wave, current, wind, ice, earthquake and their combinations. This paper presented the approach to calibrate of the reliability of the offshore jacket platform, and suggested all kinds of associated load cases. The combination of loads were mainly investigated. As an example, the reliability of offshore jacket platform of QK18-1 in Bohai Bay was verified. The result shows that the reliability of the offshore jacket platform related to 25-year reference period is between 2.7 and 3.39, which is close to the conclusion of some existing researches. This method would be used more widely for the design of offshore jacket platforms.

A Concise Introduction to Models and Methods for Automated Planning

Planning is the model-based approach to autonomous behavior where the agent behavior is derived automatically from a model of the actions, sensors, and goals. The main challenges in planning are computational as all models, whether featuring uncertainty and feedback or not, are intractable in the worst case when represented in compact form. In this book, we look at a variety of models used in AI planning, and at the methods that have been developed for solving them. The goal is to provide a modern and coherent view of planning that is precise, concise, and mostly self-contained, without being shallow. For this, we make no attempt at covering the whole variety of planning approaches, ideas, and applications, and focus on the essentials. The target audience of the book are students and researchers interested in autonomous behavior and planning from an AI, engineering, or cognitive science perspective. Table of Contents: Preface / Planning and Autonomous Behavior / Classical Planning: Full Information and Deterministic Actions / Classical Planning: Variations and Extensions / Beyond Classical Planning: Transformations / Planning with Sensing: Logical Models / MDP Planning: Stochastic Actions and Full Feedback / POMDP Planning: Stochastic Actions and Partial Feedback / Discussion / Bibliography / Author's Biography

Planning and learning in security games

We present two new critical domains where security games are applied to generate randomized patrol schedules. For each setting, we present the current research that we have produced. We then propose two new challenges to build accurate schedules that can be deployed effectively in the real world. The first is a planning challenge. Current schedules cannot handle interruptions. Thus, more expressive models, that allow for reasoning over stochastic actions, are needed. The second is a learning challenge. In several security domains, data can be used to extract information about both the environment and the attacker. This information can then be used to improve the defender's strategies.

Asymptotic guarantee of success for multi-agent memetic systems

Abstract The paper introduces a stochastic model for a class of population-based global optimization meta-heuristics, that generalizes existing models in the following ways. First of all, an individual becomes an active software agent characterized by the constant genotype and the meme that may change during the optimization process. Second, the model embraces the asynchronous processing of agent’s actions. Third, we consider a vast variety of possible actions that include the conventional mixing operations (e.g. mutation, cloning, crossover) as well as migrations among demes and local optimization methods. Despite the fact that the model fits many popular algorithms and strategies (e.g. genetic algorithms with tournament selection) it is mainly devoted to study memetic algorithms. The model is composed of two parts: EMAS architecture (data structures and management strategies) allowing to define the space of states and the framework for stochastic agent actions and the stationary Markov chain described in terms of this architecture. The probability transition function has been obtained and the Markov kernels for sample actions have been computed. The obtained theoretical results are helpful for studying metaheuristics conforming to the EMAS architecture. The designed synchronization allows the safe, coarse-grained parallel implementation and its effective, sub-optimal scheduling in a distributed computer environment. The proved strong ergodicity of the finite state Markov chain results in the asymptotic stochastic guarantee of success, which in turn imposes the liveness of a studied metaheuristic. The Markov chain delivers the sampling measure at an arbitrary step of computations, which allows further asymptotic studies, e.g. on various kinds of the stochastic convergence

An adaptive neuro-fuzzy digital filter

This work describes the neural net as identification process, considering an adaptation that adjusts the parameters in dynamical form using a proportional condition. Neural nets have problems when the weight gains are dynamically modified in a specific environment with stochastic smooth conditions adjusting in some sense in accordance with a reference signal. This output and identification error has distribution functions described as fuzzy logic stochastic actions, building membership functions with mobile inference limits, adapting the neural net weight processes in accordance with inference mechanisms. Specifically, the identification process is based on: a) A black-box scheme, where the internal weights are unknown; b) The adaptive criterion that dynamically adjusts the internal weights using fuzzy logic stochastic properties bounded by a transition function without stability loss conditions. The adaptive filter operation was illustratively presented. Key words: Estimation, fuzzy systems, neural net, digital filter, identification.

Reinforcement learning for Golog programs with first-order state-abstraction

A special feature of programs in the action language Golog are non-deterministic constructs such as non-deterministic choice of actions or arguments. It has been shown that in the presence of stochastic actions and rewards reinforcement learning techniques can be applied to obtain optimal choices for those choice-points. In order to avoid an explosion of the state space, an abstraction mechanism is employed that computes first-order state descriptions for the given program. Intuitively, the idea is to generate abstract descriptions that group together states for which the expected reward of executing the program is the same. A current limitation is that a non-deterministic choice of arguments can be handled only if the possible candidates are known in advance. In this article we show how this restriction can be lifted. We also show how a first-order variant of binary decision diagrams can be used to efficiently compute first-order state abstractions. Moreover, we give a completely declarative specification of a learning Golog interpreter that incorporates the presented state-abstraction mechanisms.

CONTINUOUS Q-LEARNING FOR MULTI-AGENT COOPERATION

The structure of the proposed Q-learning with a continuous actions set, as well as the states, consists of three parts: a fuzzy quantizer, whose output is used to learn the action policy; an action evaluation module, which models and produces the expected evaluation signal; and a stochastic action selection unit that generates an action with the expectation of better performance using a probability distribution function to estimate an optimal action selection policy. Further, the algorithm is applied to cooperative tasks where two robots must consider their partner's action before taking their own actions. Conventional Q-learning requires a predefined and discrete state space but fails to identify the variances in different situations in the same state. The proposed Q-learning with the stochastic recording real-valued unit can differentiate the actions corresponding to different state inputs but categorized to the same state. Therefore, this unit can be regarded as an action evaluation module, which models and produces the expected evaluation signal, and an action selection unit that generates an action with the expectation of better performance using a probability distribution function to estimate an optimal action selection policy. The results from the simulations demonstrate better performance and applicability of the proposed learning model.

Thermodynamics of Precision Diamond Lapping of Ceramic Surfaces

This paper reports on a methodology of designing high-precision machining of ceramic materials based on the thermodynamics of the interaction of the technological environment and the work surface, with account of changes in surface properties as a result of the stochastic action of abrasive media.

Series System Failure Rate Modeling and the Mechanism to Yield Bathtub Curve

Failure rate functions were usually constructed in terms of life distribution. The present paper defines “discrete failure rate”, analyzes the dependence of failure rate curve shape on the probabilistic property of load-strength interaction, and develops load-strength-interaction based system failure rate model. For the majority of mechanical equipment, operation load is normally a random process, components are subjected to a large number of stochastic load actions during its service life. Meanwhile, material property degrades continuously owing to the failure mechanisms such as fatigue. It is the ever changing load-strength relationship that yields the variation in failure rate. Based on the criterion that failure occurs when a load higher than the relevant strength appears, failure rate model is developed based on the interaction mechanism between load and residual strength, and their statistical characteristics as well. Based on the interpretation to the variation of failure rate in terms of load uncertainty and strength uncertainty, the effect of statistical risk of load and the effect of strength degradation on failure rate curve shape are highlighted.

Nonlinear vibrations and stability of elastic and viscoelastic systems under random stationary loads

The paper deals with numerical analysis of nonlinear vibrations of viscoelastic systems under a stochastic action in the form of a Gaussian stationary process with rational spectral density. The analysis is based on numerical simulation of the original stationary process, numerical solution of the differential equations describing the motion of the system, and computation of the maximum Lyapunov exponent if the stability of this motion is studied. An example of a plate subjected to a random stationary load applied in its plane is used to consider specific issues concerning the application of the proposed method and the peculiarities of the behavior of geometrically nonlinear elastic and viscoelastic stochastic systems. Special attention is paid to the interaction of a deterministic periodic action and a stochastic action from the viewpoint of stability of the system motion. It is shown that in some cases imposing a “colored” noise may stabilize an unstable system subjected to a periodic load.

On Lossless Approximations, the Fluctuation- Dissipation Theorem, and Limitations of Measurements

In this paper, we take a control-theoretic approach to answering some standard questions in statistical mechanics, and use the results to derive limitations of classical measurements. A central problem is the relation between systems which appear macroscopically dissipative but are microscopically lossless. We show that a linear system is dissipative if, and only if, it can be approximated by a linear lossless system over arbitrarily long time intervals. Hence lossless systems are in this sense dense in dissipative systems. A linear active system can be approximated by a nonlinear lossless system that is charged with initial energy. As a by-product, we obtain mechanisms explaining the Onsager relations from time-reversible lossless approximations, and the fluctuation-dissipation theorem from uncertainty in the initial state of the lossless system. The results are applied to measurement devices and are used to quantify limits on the so-called observer effect, also called back action, which is the impact the measurement device has on the observed system. In particular, it is shown that deterministic back action can be compensated by using active elements, whereas stochastic back action is unavoidable and depends on the temperature of the measurement device.

Elements of nonequilibrium (ћ, k) dynamics at zero and finite temperatures

A method for the development of elements of nonequilibrium (ℏ, k) dynamics without the use of the Schrodinger equation is proposed. This method is based on the generalization of the Fokker-Planck and Hamilton-Jacoby equations by the successive account of the stochastic action of vacuum (quantum thermostat). It is shown that nonequilibrium wave functions in the presence of quantum-thermal diffusion in vacuum describe the approximation to the state of generalized thermal equilibrium both at zero and finite temperatures. They can be used as the basis for a universal description of transport processes.

(ħ, k)-Dynamics as some generalization of equilibrium quantum statistical mechanics

We show that the quantum statistical mechanics (QSM) describing quantum and thermal properties of objects has only the sense of a particular semiclassical approximation. We propose a more general (than QSM) micro description of objects in a heat bath taken into account a vacuum as an object environment; we call it (ħ, k)-dynamics (ħkD). We construct a new model of thermostat, namely, a quantum heat bath and study its properties including the cases of a “cold” and “thermal” vacua. We introduce a new generative operator, Schroedingerian, or stochastic action operator, and show its fundamental role in the determination such macro-quantities as internal energy, effective temperature, and effective entropy. We establish that in ħkD the ratio of effective action to effective entropy at zero temperature equals the universal constant ħ/2k. This result corresponds to experimental data taken recently under studying of a new matter state—nearly perfect fluid.

Matched Pairs, Permutation Representations, and the Bol Property

The article examines Bol loops that are constructed from matched pairs of groups, including a new simple, non-Moufang Bol loop of order 120. Certain permutation representations of the loops under consideration provide nontrivial examples of doubly stochastic action matrices. A generalization of the matched-pair loop construction yields proper Bol actions of groups.

Stochastic deformation of integrable dynamical systems and random time symmetry

We present a deformation of a class of elementary classical integrable systems using stochastic diffusion processes. This deformation applies to the solution of the associated classical Newtonian, Hamiltonian, Lagrangian, and variational problems and to the Hamilton–Jacobi method of characteristics. The underlying stochastic action functionals involve dual random times, whose expectations are connected to the new variables of the system after a canonical transformation.

Relational Partially Observable MDPs

Relational Markov Decision Processes (MDP) are a useful abstraction for stochastic planning problems since one can develop abstract solutions for them that are independent of domain size or instantiation. While there has been an increased interest in developing relational fully observable MDPs, there has been very little work on relational partially observable MDPs (POMDP), which deal with uncertainty in problem states in addition to stochastic action effects. This paper provides a concrete formalization of relational POMDPs making several technical contributions toward their solution. First, we show that to maintain correctness one must distinguish between quantification over states and quantification over belief states; this implies that solutions based on value iteration are inherently limited to the finite horizon case. Second, we provide a symbolic dynamic programing algorithm for finite horizon relational POMDPs, solving them at an abstract level, by lifting the propositional incremental pruning algorithm. Third, we show that this algorithm can be implemented using first order decision diagrams, a compact representation for functions over relational structures, that has been recently used to solve relational MDPs.

On the smallest simple, unipotent Bol loop

Finite simple, unipotent Bol loops have recently been identified and constructed using group theory. However, the purely group-theoretical constructions of the actual loops are indirect, somewhat arbitrary in places, and rely on computer calculations to a certain extent. In the spirit of revisionism, this paper is intended to give a more explicit combinatorial specification of the smallest simple, unipotent Bol loop, making use of concepts from projective geometry and quasigroup theory along with the group-theoretical background. The loop has dual permutation representations on the projective line of order 5, with doubly stochastic action matrices.

Computing Robust Plans in Continuous Domains

We define the robustness of a sequential plan as the probability that it will execute successfully despite uncertainty in the execution environment. We consider a rich notion of uncertainty over continuous domains that includes stochastic action effects, and changes to state variables due to unpredictable exogenous events. Given a characterization of this uncertainty in terms of probability distributions (e.g., Gaussian) our contributions are two-fold: First, we describe a novel approach to computing the robustness of a plan in the situation calculus, which (a) separates the projection problem from the problem of reasoning about probability, and (b) explicitly reveals the relevance and statistical independence of random variables and events (i.e., conditions that contain random variables). Then, building on this approach, we describe a forward search based planner that generates maximally robust plans, exploiting the revealed structure for speed-up. Preliminary empirical results demonstrate that our approach can realize exponential savings in both time and space compared to the classical sampling approach.