Abstract
Decision-making is assumed to be supported by model-free and model-based systems: the model-free system is based purely on experience, while the model-based system uses a cognitive map of the environment and is more accurate. The recently developed multistep decision-making task and its computational model can dissociate the contributions of the two systems and have been used widely. This study used this task and model to understand our value-based learning process and tested alternative algorithms for the model-free and model-based learning systems. The task used in this study had a deterministic transition structure, and the degree of use of this structure in learning is estimated as the relative contribution of the model-based system to choices. We obtained data from 29 participants and fitted them with various computational models that differ in the model-free and model-based assumptions. The results of model comparison and parameter estimation showed that the participants update the value of action sequences and not each action. Additionally, the model fit was improved substantially by assuming that the learning mechanism includes a forgetting process, where the values of unselected options change to a certain default value over time. We also examined the relationships between the estimated parameters and psychopathology and other traits measured by self-reported questionnaires, and the results suggested that the difference in model assumptions can change the conclusion. In particular, inclusion of the forgetting process in the computational models had a strong impact on estimation of the weighting parameter of the model-free and model-based systems.
Highlights
Computational models are tools used to understand decision-making processes
These models differ in the combination of the basic model and the forgetting process and were denoted as the P, P-F0, P-F05, P-free default value parameter (FD), learning-rate adjustment (LA), LA-F0, LA-F05, and LA-FD models
Sensitivity to the Previous Outcome and the Weighting Parameter The computational models were developed supposing that the weighting parameter w reflects use of the “model,” or the transition structure; we examined this assumption from the statistical characteristics of the obtained data
Summary
One successful model designed for this purpose was developed by Daw et al (2011) and can dissociate the contributions of two value-based learning systems to choice behavior. One such system is the model-free system in which values are incrementally learned through direct experience. The relative contributions of these systems have been estimated by applying computational models or logistic regression models to data from the two-step decision task (Daw et al, 2011). The framework of the two learning systems has Parsimonious Learning Algorithm With Forgetting enabled productive discussions and has helped to construct theories in cognitive, psychopathological, and neuroscience research by revealing developmental changes in model-based weight (Decker et al, 2016), the predominance of model-free choice under certain circumstances (Otto et al, 2013), the neural basis of the model-free and model-based systems (Smittenaar et al, 2013; Daw and Dayan, 2014; Lee et al, 2014) and the relationships of the two systems with clinical symptoms such as those observed in obsessive-compulsive disorder (OCD) (Voon et al, 2015; Gillan et al, 2016)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
