States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.

Jan Gläscher; Nathaniel Daw; Peter Dayan; John P O'Doherty

doi:10.1016/j.neuron.2010.04.016

States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.

Standard

States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. / Gläscher, Jan; Daw, Nathaniel; Dayan, Peter; O'Doherty, John P.

in: NEURON, Jahrgang 66, Nr. 4, 4, 27.05.2010, S. 585-595.

Publikationen: SCORING: Beitrag in Fachzeitschrift/Zeitung › SCORING: Zeitschriftenaufsatz › Forschung › Begutachtung

Harvard

Gläscher, J, Daw, N, Dayan, P & O'Doherty, JP 2010, 'States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.', NEURON, Jg. 66, Nr. 4, 4, S. 585-595. https://doi.org/10.1016/j.neuron.2010.04.016

APA

Gläscher, J., Daw, N., Dayan, P., & O'Doherty, J. P. (2010). States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. NEURON, 66(4), 585-595. [4]. https://doi.org/10.1016/j.neuron.2010.04.016

Vancouver

Gläscher J, Daw N, Dayan P, O'Doherty JP. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. NEURON. 2010 Mai 27;66(4):585-595. 4. https://doi.org/10.1016/j.neuron.2010.04.016

Bibtex

@article{efa2b16992af4b6aae9097b45ca4014b,

title = "States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.",

abstract = "Reinforcement learning (RL) uses sequential experience with situations ({"}states{"}) and outcomes to assess actions. Whereas model-free RL uses this experience directly, in the form of a reward prediction error (RPE), model-based RL uses it indirectly, building a model of the state transition and outcome structure of the environment, and evaluating actions by searching this model. A state prediction error (SPE) plays a central role, reporting discrepancies between the current model and the observed state transitions. Using functional magnetic resonance imaging in humans solving a probabilistic Markov decision task, we found the neural signature of an SPE in the intraparietal sulcus and lateral prefrontal cortex, in addition to the previously well-characterized RPE in the ventral striatum. This finding supports the existence of two unique forms of learning signal in humans, which may form the basis of distinct computational strategies for guiding behavior.",

keywords = "Adolescent, Adult, Choice Behavior, Female, Forecasting, Humans, Learning, Male, Models, Neurological, Psychomotor Performance, Reinforcement (Psychology), Research Design, Reward, Young Adult",

author = "Jan Gl{\"a}scher and Nathaniel Daw and Peter Dayan and O'Doherty, {John P}",

year = "2010",

month = may,

day = "27",

doi = "10.1016/j.neuron.2010.04.016",

language = "English",

volume = "66",

pages = "585--595",

journal = "NEURON",

issn = "0896-6273",

publisher = "Cell Press",

number = "4",

}

RIS

TY - JOUR

T1 - States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning.

AU - Gläscher, Jan

AU - Daw, Nathaniel

AU - Dayan, Peter

AU - O'Doherty, John P

PY - 2010/5/27

Y1 - 2010/5/27

N2 - Reinforcement learning (RL) uses sequential experience with situations ("states") and outcomes to assess actions. Whereas model-free RL uses this experience directly, in the form of a reward prediction error (RPE), model-based RL uses it indirectly, building a model of the state transition and outcome structure of the environment, and evaluating actions by searching this model. A state prediction error (SPE) plays a central role, reporting discrepancies between the current model and the observed state transitions. Using functional magnetic resonance imaging in humans solving a probabilistic Markov decision task, we found the neural signature of an SPE in the intraparietal sulcus and lateral prefrontal cortex, in addition to the previously well-characterized RPE in the ventral striatum. This finding supports the existence of two unique forms of learning signal in humans, which may form the basis of distinct computational strategies for guiding behavior.

AB - Reinforcement learning (RL) uses sequential experience with situations ("states") and outcomes to assess actions. Whereas model-free RL uses this experience directly, in the form of a reward prediction error (RPE), model-based RL uses it indirectly, building a model of the state transition and outcome structure of the environment, and evaluating actions by searching this model. A state prediction error (SPE) plays a central role, reporting discrepancies between the current model and the observed state transitions. Using functional magnetic resonance imaging in humans solving a probabilistic Markov decision task, we found the neural signature of an SPE in the intraparietal sulcus and lateral prefrontal cortex, in addition to the previously well-characterized RPE in the ventral striatum. This finding supports the existence of two unique forms of learning signal in humans, which may form the basis of distinct computational strategies for guiding behavior.

KW - Adolescent

KW - Adult

KW - Choice Behavior

KW - Female

KW - Forecasting

KW - Humans

KW - Learning

KW - Male

KW - Models, Neurological

KW - Psychomotor Performance

KW - Reinforcement (Psychology)

KW - Research Design

KW - Reward

KW - Young Adult

U2 - 10.1016/j.neuron.2010.04.016

DO - 10.1016/j.neuron.2010.04.016

M3 - SCORING: Journal article

C2 - 20510862

VL - 66

SP - 585

EP - 595

JO - NEURON

JF - NEURON

SN - 0896-6273

IS - 4

M1 - 4

ER -