Stochastic Update Research Articles

Impact of deterministic and stochastic updates on network reciprocity in the prisoner's dilemma game

In 2 × 2 prisoner's dilemma games, network reciprocity is one mechanism for adding social viscosity, which leads to cooperative equilibrium. This study introduced an intriguing framework for the strategy update rule that allows any combination of a purely deterministic method, imitation max (IM), and a purely probabilistic one, pairwise Fermi (Fermi-PW). A series of simulations covering the whole range from IM to Fermi-PW reveals that, as a general tendency, the larger fractions of stochastic updating reduce network reciprocity, so long as the underlying lattice contains no noise in the degree of distribution. However, a small amount of stochastic flavor added to an otherwise perfectly deterministic update rule was actually found to enhance network reciprocity. This occurs because a subtle stochastic effect in the update rule improves the evolutionary trail in games having more stag-hunt-type dilemmas, although the same stochastic effect degenerates evolutionary trails in games having more chicken-type dilemmas. We explain these effects by dividing evolutionary trails into the enduring and expanding periods defined by Shigaki et al. [Phys. Rev. E 86, 031141 (2012)].

An Efficient Stochastic Update Propagation Method in Data Warehousing

This article develops a stochastic update propagation method for an operational data store (ODS) in data warehousing (DW) environments where data storage (and retrieval) is required as a sum of data at distributed source nodes. The authors' proposed method results in less network traffic (as compared with the real-time method) due to update propagation required because of changes in source data. More importantly, the method allows system users to place limits on the discrepancy between the source data and the ODS data that could result due to a time lag between source data changes and the update operation. Finally, the pre-specified limits on the discrepancy are maintained while accounting for two crucial factors in distributed systems: 1) some nodes are situated on more congested network links, and 2) some of the links on the network are less reliable. Real-time data propagation does not account for these frequently encountered networking concerns.

Controllability of Boolean control networks under asynchronous stochastic update with time delay

In this article, the controllability of asynchronous Boolean control networks (ABCNs) with time-invariant delay is studied. Using semi-tensor product, Boolean control networks under Harvey’s asynchronous update with time delay are converted into linear representation, and a general formula of control-depending network transition matrices is achieved. Firstly, a necessary and sufficient condition is proposed to verify that only control-depending fixed points of ABCNs can be deterministically controlled. Secondly, respectively for three kinds of controls, the controllability of the non-deterministic logical dynamics is investigated, and formulas are given to show: a) the reachable sets at time s under a group of specified controls; b) the reachable sets at time s under arbitrary controls and c) the probability values from a given initial state to different target states. Moreover, we also discuss how to find the particular control sequence to reach a specified target state with the maximum probability. Examples are shown to illustrate the feasibility of the proposed scheme.

On Gaussian filters for continuous-discrete nonlinear systems

This paper considers a continuous-discrete (CD) nonlinear filtering problem in the framework of Gaussian filters. After reviewing the equivalent linearization of static nonlinearities, we derive the continuous-discrete exact Gaussian filter (CD-ExGF) and equivalent linearization Kalman filter (CD-EqKF). It is shown that two CD nonlinear filters have the same time update differential equations for the conditional mean and covariance matrix, and that a difference is in the Kalman gain for the measurement update. Numerical methods of integrating a stochastic differential equation and time update differential equations are developed using the Heun scheme. Results of two simulation studies are included to show the difference and similarity of the CD-EqKF, CD-ExGF and CD extended-Kalman filter (CD-EKF).

Opportunistic migration in spatial evolutionary games

We study evolutionary games in a spatial diluted grid environment in which agents strategically interact locally but can also opportunistically move to other positions within a given migration radius. Using the imitation of the best rule for strategy revision, it is shown that cooperation may evolve and be stable in the Prisoner's Dilemma game space for several migration distances but only for small game interaction radius while the Stag Hunt class of games become fully cooperative. We also show that only a few trials are needed for cooperation to evolve, i.e., searching costs are not an issue. When the stochastic Fermi strategy update protocol is used cooperation cannot evolve in the Prisoner's Dilemma if the selection intensity is high in spite of opportunistic migration. However, when imitation becomes more random, fully or partially cooperative states are reached in all games for all migration distances tested and for short to intermediate interaction radii.

Vague-to-crisp dynamics of percept formation modeled as operant (selectionist) process

We model the vague-to-crisp dynamics of forming percepts in the brain by combining two methodologies: dynamic logic (DL) and operant learning process. Forming percepts upon the presentation of visual inputs is likened to model selection based on sampled evidence. Our framework utilizes the DL in selecting the correct "percept" among competing ones, but uses an intrinsic reward mechanism to allow stochastic online update in lieu of performing the optimization step of the DL framework. We discuss the connection of our framework with cognitive processing and the intentional neurodynamic cycle.

Dynamics of Random Boolean Networks under Fully Asynchronous Stochastic Update Based on Linear Representation

A novel algebraic approach is proposed to study dynamics of asynchronous random Boolean networks where a random number of nodes can be updated at each time step (ARBNs). In this article, the logical equations of ARBNs are converted into the discrete-time linear representation and dynamical behaviors of systems are investigated. We provide a general formula of network transition matrices of ARBNs as well as a necessary and sufficient algebraic criterion to determine whether a group of given states compose an attractor of length in ARBNs. Consequently, algorithms are achieved to find all of the attractors and basins in ARBNs. Examples are showed to demonstrate the feasibility of the proposed scheme.

A unified approach to tracking performance analysis of the selective partial update adaptive filter algorithms in nonstationary environment

In this paper, a unified approach to mean-square performance analysis of the family of selective partial update (SPU) adaptive filter algorithms in nonstationary environment is presented. Using this analysis, the tracking performance of Max normalized least mean squares (Max-NLMS), N-Max NLMS, the various types of SPU-NLMS algorithms, SPU transform domain LMS (SPU-TD-LMS), the family of SPU affine projection algorithms (SPU-APA), the family of selective regressor APA (SR-APA), the dynamic selection of APA (DS-APA), the family of SPU-SR-APA, the family of SPU-DS-APA, SPU subband adaptive filters (SPU-SAF), and the periodic, sequential, and stochastic partial update LMS, NLMS, and APA as well as classical adaptive filter algorithms can be analyzed with a unified approach. Two theoretical expressions are introduced to study the performance. The analysis is based on energy conservation arguments and does not need to assume a Gaussian or white distribution for the regressors. We demonstrate through simulations that the derived expressions are useful in predicting the performance of this family of adaptive filters in nonstationary environment.

A mixed-discrete Particle Swarm Optimization algorithm with explicit diversity-preservation

Engineering design problems often involve non-linear criterion functions, including inequality and equality constraints, and a mixture of discrete and continuous design variables. Optimization approaches entail substantial challenges when solving such an all-inclusive design problem. In this paper, a modification of the Particle Swarm Optimization (PSO) algorithm is presented, which can adequately address system constraints while dealing with mixed-discrete variables. Continuous search (particle motion), as in conventional PSO, is implemented as the primary search strategy; subsequently, the discrete variables are updated using a deterministic nearest-feasible-vertex criterion. This approach is expected to alleviate the undesirable difference in the rates of evolution of discrete and continuous variables. The premature stagnation of candidate solutions (particles) due to loss of diversity is known to be one of the primary drawbacks of the basic PSO dynamics. To address this issue in high dimensional design problems, a new adaptive diversity-preservation technique is developed. This technique characterizes the population diversity at each iteration. The estimated diversity measure is then used to apply (i) a dynamic repulsion away from the best global solution in the case of continuous variables, and (ii) a stochastic update of the discrete variables. For performance validation, the Mixed-Discrete PSO algorithm is applied to a wide variety of standard test problems: (i) a set of 9 unconstrained problems, and (ii) a comprehensive set of 98 Mixed-Integer Nonlinear Programming (MINLP) problems. We also explore the applicability of this algorithm to a large scale engineering design problem—-wind farm layout optimization.

Fixed-Energy Sandpiles Belong Generically to Directed Percolation

Fixed-energy sandpiles with stochastic update rules are known to exhibit a nonequilibrium phase transition from an active phase into infinitely many absorbing states. Examples include the conserved Manna model, the conserved lattice gas, and the conserved threshold transfer process. It is believed that the transitions in these models belong to an autonomous universality class of nonequilibrium phase transitions, the so-called Manna class. Contrarily, the present numerical study of selected (1+1)-dimensional models in this class suggests that their critical behavior converges to directed percolation after very long time, questioning the existence of an independent Manna class.

Evaluation of attractors and basins of asynchronous random Boolean networks

We present an algebraic approach for determining the attractors and their basins of random Boolean networks under an asynchronous stochastic update based on the recently developed matrix semitensor product theory, which allows for converting the logical dynamics of a Boolean network into a standard iterative dynamics. In this setting, all attractors and basins are specified by the network transition matrices. We then devise procedures that can find all attractors and their basins exactly. We also discuss the issue of overlapping basins in asynchronous random Boolean networks, and we propose methods to compute the weight of each attractor and the basin entropy of the systems.

Transient Analysis of Adaptive Affine Combinations

In this correspondence, we provide a transient analysis of an affinely constrained mixture method that adaptively combines the outputs of adaptive filters running in parallel on the same task. The affinely constrained mixture is adapted using a stochastic gradient update to minimize the square of the prediction error. Although we specifically carry out the transient analysis for a combination of two equal length adaptive filters trying to learn a linear model working on real valued data, we also provide the final equations and the necessary extensions in order to generalize the transient analysis to mixtures combining more than two filters; using Newton based updates to train the mixture weights; working on complex valued data; or unconstrained mixtures. The derivations are generic such that the constituent filters can be trained using unbiased updates including the least-mean squares or recursive least squares updates. This correspondence concludes with numerical examples and final remarks.

Optimal Stochastic Location Updates in Mobile Ad Hoc Networks

We consider the location service in a mobile ad-hoc network (MANET), where each node needs to maintain its location information by 1) frequently updating its location information within its neighboring region, which is called neighborhood update (NU), and 2) occasionally updating its location information to certain distributed location server in the network, which is called location server update (LSU). The trade off between the operation costs in location updates and the performance losses of the target application due to location inaccuracies (i.e., application costs) imposes a crucial question for nodes to decide the optimal strategy to update their location information, where the optimality is in the sense of minimizing the overall costs. In this paper, we develop a stochastic sequential decision framework to analyze this problem. Under a Markovian mobility model, the location update decision problem is modeled as a Markov Decision Process (MDP). We first investigate the monotonicity properties of optimal NU and LSU operations with respect to location inaccuracies under a general cost setting. Then, given a separable cost structure, we show that the location update decisions of NU and LSU can be independently carried out without loss of optimality, i.e., a separation property. From the discovered separation property of the problem structure and the monotonicity properties of optimal actions, we find that 1) there always exists a simple optimal threshold-based update rule for LSU operations; 2) for NU operations, an optimal threshold-based update rule exists in a low-mobility scenario. In the case that no a priori knowledge of the MDP model is available, we also introduce a practical model-free learning approach to find a near-optimal solution for the problem.

Stochastic Characterization and Update of Fatigue Loading for Mechanical Damage Prognosis

Accurate characterization and prediction of loading, while properly accounting for uncertainty, are essential for probabilistic fatigue damage prognosis. Three different techniques, including rainflow counting, the Markov chain method, and autoregressive moving average (ARMA) method, are reviewed for stochastic characterization and reconstruction of the fatigue load spectrum for prognosis. The ARMA method is extended by introducing random coefficients and probabilistic weights, to account for the uncertainty in the selection of models, inherent variability of loading, and uncertainty due to sparse data. A continuous model updating framework based on usage monitoring data is developed and applied to all the three techniques mentioned above. The relation between prediction accuracy and updating period is evaluated quantitatively. A quantitative model validation metric is proposed for assessing the accuracy of load prediction.*

Co-evolution of strategies and update rules in the prisoner's dilemma game on complex networks

In this paper, we study a weak prisoner's dilemma (PD) game in which both strategies and update rules are subjected to evolutionary pressure. Interactions among agents are specified by complex topologies, and we consider both homogeneous and heterogeneous situations. We consider deterministic and stochastic update rules for the strategies, which in turn may consider single links or the full context when selecting agents to copy from. Our results indicate that the co-evolutionary process preserves heterogeneous networks as a suitable framework for the emergence of cooperation. Furthermore, on those networks the update rule leading to a larger fraction, which we call replicator dynamics, is selected during co-evolution. On homogeneous networks, we observe that even if the replicator dynamics again turns out to be the selected update rule, the cooperation level is greater than on a fixed update rule framework. We conclude that for a variety of topologies, the fact that the dynamics co-evolves with the strategies leads, in general, to more cooperation in the weak PD game.

Attractor and basin entropies of random Boolean networks under asynchronous stochastic update

We introduce a method to study random Boolean networks with asynchronous stochastic update. Each node in the state space network starts with equal occupation probability, which then evolves to a steady state. Attractors and the sizes of their basins are determined by the nodes left occupied at late times. As for synchronous update, the basin entropy grows with system size only for critical networks. We determine analytically the distribution for the number of attractors and basin sizes for networks with connectivity K=1 . These differ from the case of synchronous update for all K .

Sequential and Cooperative Sensing for Multi-Channel Cognitive Radios

Effective spectrum sensing is a critical prerequisite for multi-channel cognitive radio (CR) networks, where multiple spectrum bands are sensed to identify transmission opportunities, while preventing interference to the primary users. The present paper develops sequential spectrum sensing algorithms which explicitly take into account the sensing time overhead, and optimize a performance metric capturing the effective average data rate of CR transmitters. A constrained dynamic programming problem is formulated to obtain the policy that chooses the best time to stop taking measurements and the best set of channels to access for data transmission, while adhering to hard “collision” constraints imposed to protect primary links. Given the associated Lagrange multipliers, the optimal access policy is obtained in closed form, and the subsequent problem reduces to an optimal stopping problem. A basis expansion-based sub-optimal strategy is employed to mitigate the prohibitive computational complexity of the optimal stopping policy. A novel on-line implementation based on the recursive least-squares (RLS) algorithm along with a stochastic dual update procedure is then developed to obviate the lengthy training phase of the batch scheme. Cooperative sequential sensing generalizations are also provided with either raw or quantized measurements collected at a central processing unit. The numerical results presented verify the efficacy of the proposed algorithms.

Stochastic meshfree method for elasto-plastic damage analysis

This paper presents a stochastic meshfree method for solving boundary -value problems in damage mechanics under elasto -plastic conditions. Isotropic ductile damage evolution law is used to model the coupled elasto -plastic damage growth. Uncertainty associated with initial damage in materials is considered as random field. Moving least squares shape function method and Karhunen Loève expansion method are used for random field discretization. Statistical parameters of the response quantities are computed using perturbation method. The proposed method involves a new stochastic stress update procedure to solve the nonlinear equations in terms of discretized random variables arising from perturbation of equilibrium equations system. Numerical examples comprising of one and two dimensional problems are presented to illustrate the effectiveness of proposed method.

Space representation of stochastic processes with delay

We show that a time series x(t) evolving by a nonlocal update rule x(t) =f (x(t-n),x(t-k)) with two different delays k < n can be mapped onto a local process in two dimensions with special time-delayed boundary conditions, provided that n and k are coprime. For certain stochastic update rules exhibiting a nonequilibrium phase transition, this mapping implies that the critical behavior does not depend on the short delay k . In these cases, the autocorrelation function of the time series is related to the critical properties of the corresponding two-dimensional model.

Scope of stationary multi-objective evolutionary optimization: a case study on a hydro-thermal power dispatch problem

Many engineering design and developmental activities finally resort to an optimization task which must be solved to get an efficient and often an intelligent solution. Due to various complexities involved with objective functions, constraints, and decision variables, optimization problems are often not adequately suitable to be solved using classical point-by-point methodologies. Evolutionary optimization procedures use a population of solutions and stochastic update operators in an iteration in a manner so as to constitute a flexible search procedure thereby demonstrating promise to such difficult and practical problem-solving tasks. In this paper, we illustrate the power of evolutionary optimization algorithms in handling different kinds of optimization tasks on a hydro-thermal power dispatch optimization problem: (i) dealing with non-linear, non-differentiable objectives and constraints, (ii) dealing with more than one objectives and constraints, (iii) dealing with uncertainties in decision variables and other problem parameters, and (iv) dealing with a large number (more than 1,000) variables. The results on the static power dispatch optimization problem are compared with that reported in an existing simulated annealing based optimization procedure on a 24-variable version of the problem and new solutions are found to dominate the solutions of the existing study. Importantly, solutions found by our approach are found to satisfy theoretical Kuhn---Tucker optimality conditions by using the subdifferentials to handle non-differentiable objectives. This systematic and detail study demonstrates that evolutionary optimization procedures are not only flexible and scalable to large-scale optimization problems, but are also potentially efficient in finding theoretical optimal solutions for difficult real-world optimization problems.