Habits, Action Sequences And Working Memory From A Behavioral And A Computational Perspective

Abstract

Event Abstract Back to Event Habits, Action Sequences And Working Memory From A Behavioral And A Computational Perspective Vincent Moens1*, Alexandre Zénon1 and Etienne OLIVIER1 1 UCLouvain, Belgium Habitual and goal-directed behaviors have long been investigated (1–3). Habits are extensively trained behaviors that are efficient in a stable environment and insensitive to contingency degradation or change of the associative structure (CAS) of the environment (4–6). In contrast, goal-directed actions are more efficient in a new or ever-changing environment, but are subject to interference with dual tasks, presumably because they are more computationally demanding. The exact computational and neural correlates of these two types of behaviors remain debated. The now “classical” view is that a model-free reinforcement-learning algorithm could reflect habitual control whereas a model-based reinforcement-learning algorithm could reflect the goal-directed control (7–13). However, other additional factors are also likely to influence the selection or reliance on habitual versus goal-directed behaviors. The goal-directed controller is also supposed to lose influence as habits gain in reliability, but other factors, such as working memory (WM) capacity and WM loading, could influence the weighting between the two systems (14,15). It is also unknown if learning of action sequences could, in whole or in part, account for habit acquisition as opposed to the step-by-step, single action-stimulus association formation, as assumed by the classical theory (16–18). The goal of the present study was to investigate (1) if action sequence formation could account at least in part for habit acquisition, and if these action sequences were procedural or perceptual in nature (2) if WM loading with respect to WM capacity could reflect the propensity of switching from goal-directed to habitual controller and (3) if pupil dilation could reflect the weight given to the habitual or goal-directed controller of action selection. Here we used a two-step task aimed at inducing habitual behavior through extensive training in a stable environment. Participants (n=17) had to perform successively two different actions in response to four visual stimuli in order to reach one of the four desired final state associated with each stimulus. Actions consist of key pressing on a computer keyboard and a mouse. Action sequences and stimuli were divided in two different groups (i.e. stimuli 1 and 2 required action sequences 1 and 2 whereas stimuli 3 and 4 required action sequences 3 and 4). At first, each of those action sequences had a fixed frequency, defined independently for each stimulus (0% and 100% for stimulus 1, 50% and 50% for stimulus 2, 17% and 83% for stimulus 3 and 33% and 67% for stimulus 4). Subjects were not aware of those frequencies. The experiment was achieved in two sessions on two successive days and each subject performed a total of 1200 trials in approximately 150 min. After a training period of 700 trials (175 trials per stimulus), a WM loading task was added in parallel to the main task (Stroop task). After a short retraining in this condition (175 trials), the contingency of the system was changed for the last 325 trials and each shape stimulus was henceforth associated with all the possible action sequences with an equal probability. WM was assessed at the beginning of the experiment with a spatial updating task. Pupil size dilation was also measured during all the experiment. We expected the habitual behavior (i.e. the difficulty to adapt to change) to be proportional to the probability of the associated/corresponding action sequence: stimulus trained with a 100%-0% action sequence pair should be more habitual than the 83%-17% pair, the 83%-17% more habitual than the 67%-33% pair and so on. Preliminary results show that the performance after the CAS depended of the strength of the association between the stimuli and the corresponding action sequence. They also show that WM capacity is correlated to the overall performance and to the ability to adapt to the new contingency after the CAS. We tested three computational models on the present task that tested the presence or the absence of perceptual and procedural action sequence controllers. The most complex model was significantly more accurate than the simpler ones and it helped to disentangle the contribution of WM in the action selection process. The pupil size measurements were then analyzed in the perspective of those models. Acknowledgements Computational resources have been provided by the supercomputing facilities of the Université Catholique de Louvain (CISM/UCL).