Lose-shift Strategy Research Articles

Slow emergence of cooperation for win-stay lose-shift on trees

We consider a group of agents on a graph who repeatedly play the prisoner’s dilemma game against their neighbors. The players adapt their actions to the past behavior of their opponents by applying the win-stay lose-shift strategy. On a finite connected graph, it is easy to see that the system learns to cooperate by converging to the all-cooperate state in a finite time. We analyze the rate of convergence in terms of the size and structure of the graph. Dyer et al. (2002) showed that the system converges rapidly on the cycle, but that it takes a time exponential in the size of the graph to converge to cooperation on the complete graph. We show that the emergence of cooperation is exponentially slow in some expander graphs. More surprisingly, we show that it is also exponentially slow in bounded-degree trees, where many other dynamics are known to converge rapidly.

The efficiency of adapting aspiration levels

Win–stay, lose–shift strategies in repeated games are based on an aspiration level. A move is repeated if and only if the outcome, in the previous round, was satisficing in the sense that the pay-off was at least as high as the aspiration level. We investigate the conditions under which adaptive mechanisms acting on the aspiration level (selection, for instance, or learning) can lead to an efficient outcome; in other words, when can satisficing become optimizing? Analytical results for 2 times 2-games are presented. They suggest that in a large variety of social interactions, self-centred rules (based uniquely on one's own pay-off) cannot suffice.

Win–Stay, Lose–Shift Strategies for Repeated Games—Memory Length, Aspiration Levels and Noise

Win–stay, lose–shift, the principle to retain a successful action is a simple and general learning rule that can be applied to all types of repeated decision problems. In this paper I consider win–stay, lose–shift strategies with diverse memory sizes and strategies that adapt their aspiration levels, i.e. the payoff level considered as "success'’. I study their evolution for the Prisoner's Dilemma, as well as in a rapidly changing environment, where a randomly selected game is assigned to the players. For win–stay, lose–shift strategies with memory one the average payoffs are computed and their evolutionary stability is discussed. Using computer simulations I show that the win–stay, lose–shift strategies with longer memory are very successful both for the Prisoner's Dilemma, where cooperation dominates even for high noise levels, and the randomly assigned games, where the players achieve nearly the expected Pareto optimal payoffs. I discuss the impact of noise and show that the memory length of the players increases with the noise level. These results indicate that the win–stay, lose–shift principle is a very successful strategy in repeated games with noise.

Miocene intra‐arc bending at an arc–arc collision zone, central Japan: Comment

Kanazawa (2000) asks a familiar question in the rational choice theory of politics: Why would instrumentally rational actors vote when they have nil probability of influencing the election results? Kanazawa argues that citizens estimate their probability of influencing the election not from the number of voters, but from their own past voting choices and previous election outcomes. In Kanazawa's stochastic learning model, potential voters follow a win-stay/ lose-shift behavior strategy that predicts distinctive voting patterns across presidential elections. Kanazawa claims to find such voting patterns in data from a series of General Social Surveys. We replicate Kanazawa's findings and address two methodological concerns. First, Kanazawa's models contain numerous coding irregularities. Some of the irregularities are a deliberate but unorthodox treatment of ambiguous voting responses, while others are simple mistakes that miscode, drop, or inappropriately retain cases. Second, Kanazawa removes two of seven surveys from the sample without (we argue) a compelling justification. We reestimate Kanazawa's model with correctly coded variables and the full sample and find no evidence for stochastic learning in voting turnout.

Ginseng improves strategic learning by normal and brain-damaged rats.

Adult rats were prepared with either sham or medial prefrontal cortex lesions and administered, beginning on the third post-operative day, either, 0, 40, or 80 mg kg-1 crude ginseng extract suspended in saline daily for the next 30 days. Later, kinetic functions were evaluated on an elevated rotating rod. No long-term influences of the treatments were observed on this task. Significant positive influences of ginseng were observed in the position reversal task. The learning deficits observed in the saline control brain-damaged rats were significantly attenuated in the ginseng-treated animals. An analysis of trial 2 response accuracy across reversals revealed enhanced cognitive abilities (i.e. acquisition of a win-stay, lose-shift strategy) in both the brain damaged and sham control rats administered ginseng. Generally, administration of the higher dose resulted in better performance in the learning paradigm. The exact mechanism responsible for these promising results remains to be discovered. Several possible mechanisms are discussed.

Cognitive strategies and hypothesis testing during discrimination learning in Parkinson's disease

Rhis study investigated the nature and extent of impairments in the use of hypotheses and cognitive strategies in medicated subjects with idiopathic Parkinson's disease (PD) and matched control subjects. PD subjects did not differ from controls in solving one- or two-dimensional discrimination learning problems, but showed impairment on four-dimensional problems which did not appear to be attributable to memory deficits. They achieved fewer correct solutions, used fewer hypotheses, and were less likely to use appropriate lose-shift strategies following negative feedback. The pattern of findings was similar to those previously reported for subjects with frontal lobe lesions.

The Nazi Doctors

<h3>Abstract</h3> Zebrafish (<i>Danio rerio</i>) constitute an excellent model system to investigate the neural and genetic basis of quantitative cognition because of the single neuron resolution of calcium imaging of awake, behaving fish. While nonsymbolic numerical cognition has been investigated across many taxa, symbolic numerical cognition has not been investigated among fish. We developed a novel quantitative symbolic test for zebrafish using an operant conditioning paradigm in which the number of horizontal lines zebrafish approached in a 2-alternative forced choice task predicted the number of food reward pellets they would receive. Zebrafish did not at the population level learn a preference for the 2-line stimulus predictive of receiving 2 food pellets. However, they performed significantly above chance in a nonsymbolic discrimination task with the same apparatus, in which the 2-line stimulus was associated with the same reward but the choice of the 1-line stimulus was not rewarded. We also explored the explanatory value of alternative spatial learning hypotheses such as a Win-Stay, Lose-Shift (WSLS) strategy at the individual level for fish in navigating these spatially randomised tasks. The implications of this for symbolic versus nonsymbolic quantitative cognition in this model system are discussed relative to reward type and stimulus modality.

Interproblem Learning in Ten-Month-Old Infants

FAGEN, JEFFREY W. Interproblem Learning in Ten-Month-Old Infants. Cmun DEVELOPMENT, 1977, 48, 786-796. The ability of 10-month-old infants to acquire an object discrimination was assessed through the learning-set task. 4 infants received 2 sessions per day for 10 consecutive days. In the first 2 sessions (Day 1), they received a single 84-trial discrimination problem. Each of the remaining 18 sessions (Days 2-10) consisted of 7 6-trial problems, each with a different pair of discriminative stimuli. On Day 1, subjects achieved a criterion of 10 consecutive correct responses in a mean of 69.75 trials. No subject reached a learning-set criterion of no more than 1 error on Trials 2-6 of 4 consecutive problems in a single session. Interproblem learning was evidenced by a linear trend in the number of correct Trial 2-6 responses over the 9 learning-set days. Trial 2 performance was reliably above that of Trial 1 during the last half of training only. Infants were found to adopt a partial win-stay, lose-shift response strategy which was especially apparent on problems in which their response latency was above the median latency to respond. Criterion performance appeared to be prevented by a strong position preference present in every infant.

SHORT‐TERM MEMORY IN THE PIGEON: THE PREVIOUSLY REINFORCED RESPONSE1

Eighteen pigeons served in a discrete-trials short-term memory experiment in which the reinforcement probability for a peck on one of two keys depended on the response reinforced on the previous trial: either the probability of reinforcement on a trial was 0.8 for the same response reinforced on the previous trial and was 0.2 for the other response (Group A), or, it was 0 or 0.2 for the same response and 1.0 or 0.8 for the other response (Group B). A correction procedure ensured that over all trials reinforcement was distributed equally across the left and right keys. The optimal strategy was either a winstay, lose-shift strategy (Group A) or a win-shift, lose-stay strategy (Group B). The retention interval, that is the intertrial interval, was varied. The average probability of choosing the optimal alternative reinforced 80% of the time was 0.96, 0.84, and 0.74 after delays of 2.5, 4.0, and 6.0 sec, respectively for Group A, and was 0.87, 0.81, and 0.55 after delays of 2.5, 4.0, and 6.0 sec, respectively, for Group B. This outcome is consistent with the view that behavior approximated the optimal response strategy but only to an extent permitted by a subject's short-term memory for the cue correlated with reinforcement, that is, its own most-recently reinforced response. More generally, this result is consistent with "molecular" analyses of operant behavior, but is inconsistent with traditional "molar" analyses holding that fundamental controlling relations may be discovered by routinely averaging over different local reinforcement contingencies. In the present experiment, the molar results were byproducts of local reinforcement contingencies involving an organism's own recent behavior.

Transfer effects of simple and detailed instructions in the presence and absence of pretraining in reversal and nonreversal shifts

Two studies investigating the effects of pretraining and task instructions on transfer performance in reversal, nonreversal, and control shift paradigms were performed. In Experiment 1, all Ss received two unidimensional training problems, while task instructions and transfer paradigm were varied. In Experiment 2, task instructions, pretraining, and transfer paradigm were varied. Simple task instructions and the absence of pretraining both tend to facilitate reversal shifts and retard nonreversal shifts. Detailed task instructions and pretraining tend to facilitate nonreversal shifts and retard reversal shifts. It is suggested that a mechanism involving the presence or absence of a dimensional lose-shift strategy will account for the results.

Cognitive process in auditory concept acquisition

A comparison of the performance of 15 adult Ss who were required to verbalize hypotheses and to make category responses during concept acquisition to that of 15 Ss who made only category responses indicated that verbalization does not greatly influence the acquisition process. Tests of stationarity for the acquisition process support an all-or-nothing model of concept acquisition. A win-stay, lose-shift strategy of hypothesis testing prevailed, regardless of the complexity of the hypothesis under consideration. It was found that, in the absence of cues as to the nature of the solution concept, Ss typically first considered simple one-attribute hypotheses and then moved on to more complex two- and three-attribute hypotheses.