Rxivist logo

Rxivist combines preprints from bioRxiv with data from Twitter to help you find the papers being discussed in your field. Currently indexing 62,736 bioRxiv papers from 278,376 authors.

The successor representation in human reinforcement learning

By Ida Momennejad, Evan M. Russek, JH Cheong, Matthew M. Botvinick, Nathaniel D. Daw, Samuel Gershman

Posted 27 Oct 2016
bioRxiv DOI: 10.1101/083824 (published DOI: 10.1038/s41562-017-0180-8)

Theories of reward learning in neuroscience have focused on two families of algorithms, thought to capture deliberative vs. habitual choice. Model-based algorithms compute the value of candidate actions from scratch, whereas model-free algorithms make choice more efficient but less flexible by storing pre-computed action values. We examine an intermediate algorithmic family, the successor representation (SR), which balances flexibility and efficiency by storing partially computed action values: predictions about future events. These pre-computation strategies differ in how they update their choices following changes in a task. SR's reliance on stored predictions about future states predicts a unique signature of insensitivity to changes in the task's sequence of events, but flexible adjustment following changes to rewards. We provide evidence for such differential sensitivity in two behavioral studies with humans. These results suggest that the SR is a computational substrate for semi-flexible choice in humans, introducing a subtler, more cognitive notion of habit.

Download data

  • Downloaded 11,875 times
  • Download rankings, all-time:
    • Site-wide: 76 out of 62,736
    • In neuroscience: 12 out of 11,207
  • Year to date:
    • Site-wide: 969 out of 62,736
  • Since beginning of last month:
    • Site-wide: 3,063 out of 62,736

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide

Sign up for the Rxivist weekly newsletter! (Click here for more details.)