The hippocampus as a predictive map

Kimberly L Stachenfeld; Matthew M Botvinick; Samuel J Gershman

doi:10.1038/nn.4650

journal article Oct 02, 2017

The hippocampus as a predictive map

Kimberly L Stachenfeld Matthew M Botvinick Samuel J Gershman

Nature Neuroscience Vol. 20 No. 11 pp. 1643-1653 · Springer Science and Business Media LLC

View at Publisher Save 10.1038/nn.4650

Topics

No keywords indexed for this article. Browse by subject →

References

55

[1]

Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

Nathaniel D Daw, Yael Niv, Peter Dayan

Nature Neuroscience 2005 10.1038/nn1560

[2]

Cognitive maps in rats and men.

Edward C. Tolman

Psychological Review 1948 10.1037/h0061626

[3]

A Neural Substrate of Prediction and Reward

Wolfram Schultz, Peter Dayan, P. Read Montague

Science 1997 10.1126/science.275.5306.1593

[4]

Dayan, P. Improving generalization for temporal difference learning: the successor representation. Neural Comput. 5, 613–624 (1993). 10.1162/neco.1993.5.4.613

[5]

O'Keefe, J. & Nadel, L. The Hippocampus as a Cognitive Map (Clarendon Press, 1978).

[6]

Muller, R.U., Stead, M. & Pach, J. The hippocampus as a cognitive graph. J. Gen. Physiol. 107, 663–694 (1996). 10.1085/jgp.107.6.663

[7]

Penny, W.D., Zeidman, P. & Burgess, N. Forward and backward inference in spatial cognition. PLOS Comput. Biol. 9, e1003383 (2013). 10.1371/journal.pcbi.1003383

[8]

Foster, D.J., Morris, R.G. & Dayan, P. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10, 1–16 (2000). 10.1002/(sici)1098-1063(2000)10:1<1::aid-hipo1>3.0.co;2-1

[9]

Gustafson, N.J. & Daw, N.D. Grid cells, place cells, and geodesic generalization for spatial reinforcement learning. PLOS Comput. Biol. 7, e1002235 (2011). 10.1371/journal.pcbi.1002235

[10]

Hafting, T., Fyhn, M., Molden, S., Moser, M.B. & Moser, E.I. Microstructure of a spatial map in the entorhinal cortex. Nature 436, 801–806 (2005). 10.1038/nature03721

[11]

Gershman, S.J., Moore, C.D., Todd, M.T., Norman, K.A. & Sederberg, P.B. The successor representation and temporal context. Neural Comput. 24, 1553–1568 (2012). 10.1162/neco_a_00282

[12]

Russek, E.M., Momennejad, I., Botvinick, M.M., Gershman, S.J. & Daw, N.D. Predictive representations can link model-based reinforcement learning to model-free mechanisms. Preprint at https://doi.org/10.1101/083857 (2017). 10.1101/083857

[13]

Schapiro, A.C., Turk-Browne, N.B., Norman, K.A. & Botvinick, M.M. Statistical learning of temporal community structure in the hippocampus. Hippocampus 26, 3–8 (2016). 10.1002/hipo.22523

[14]

Dordek, Y., Meir, R. & Derdikman, D. Extracting grid characteristics from spatially distributed place cell inputs using non-negative PCA. Preprint at https://arxiv.org/abs/1505.03711 (2015). 10.7554/elife.10094

[15]

Mehta, M.R., Quirk, M.C. & Wilson, M.A. Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron 25, 707–715 (2000). 10.1016/s0896-6273(00)81072-7

[16]

The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells

RU Muller, JL Kubie

The Journal of Neuroscience 1987 10.1523/jneurosci.07-07-01951.1987

[17]

Skaggs, W.E. & McNaughton, B.L. Spatial firing properties of hippocampal CA1 populations in an environment containing two visually identical regions. J. Neurosci. 18, 8455–8466 (1998). 10.1523/jneurosci.18-20-08455.1998

[18]

Alvernhe, A., Save, E. & Poucet, B. Local remapping of place cell firing in the Tolman detour task. Eur. J. Neurosci. 33, 1696–1705 (2011). 10.1111/j.1460-9568.2011.07653.x

[19]

Hollup, S.A., Molden, S., Donnett, J.G., Moser, M.B. & Moser, E.I. Accumulation of hippocampal place fields at the goal location in an annular water maze task. J. Neurosci. 21, 1635–1644 (2001). 10.1523/jneurosci.21-05-01635.2001

[20]

Strange, B.A., Witter, M.P., Lein, E.S. & Moser, E.I. Functional organization of the hippocampal longitudinal axis. Nat. Rev. Neurosci. 15, 655–669 (2014). 10.1038/nrn3785

[21]

Garvert, M.M., Dolan, R.J. & Behrens, T.E. A map of abstract relational knowledge in the human hippocampal-entorhinal cortex. eLife 6, 17086 (2017). 10.7554/elife.17086

[22]

Deuker, L., Bellmund, J.L., Navarro Schröder, T. & Doeller, C.F. An event map of memory space in the hippocampus. eLife 5, e16534 (2016). 10.7554/elife.16534

[23]

Krupic, J., Bauza, M., Burton, S., Barry, C. & O'Keefe, J. Grid cell symmetry is shaped by environmental geometry. Nature 518, 232–235 (2015). 10.1038/nature14153

[24]

Derdikman, D. et al. Fragmentation of grid cell maps in a multicompartment environment. Nat. Neurosci. 12, 1325–1332 (2009). 10.1038/nn.2396

[25]

Carpenter, F., Manson, D., Jeffery, K., Burgess, N. & Barry, C. Grid cells form a global representation of connected environments. Curr. Biol. 25, 1176–1182 (2015). 10.1016/j.cub.2015.02.037

[26]

Mazumder, R., Hastie, T. & Tibshirani, R. Spectral regularization algorithms for learning large incomplete matrices. J. Mach. Learn. Res. 11, 2287–2322 (2010).

[27]

Mahadevan, S. & Maggioni, M. Proto-value functions: a Laplacian framework for learning representation and control in markov decision processes. J. Mach. Learn. Res. 8, 2169–2231 (2007).

[28]

Bonnevie, T. et al. Grid cells require excitatory drive from the hippocampus. Nat. Neurosci. 16, 309–317 (2013). 10.1038/nn.3311

[29]

Hales, J.B. et al. Medial entorhinal cortex lesions only partially disrupt hippocampal place cells and hippocampus-dependent place memory. Cell Rep. 9, 893–901 (2014). 10.1016/j.celrep.2014.10.009

[30]

Solway, A. et al. Optimal behavioral hierarchy. PLoS Comput. Biol. 10, e1003779 (2014). 10.1371/journal.pcbi.1003779

[31]

Ribas-Fernandes, J.J. et al. A neural signature of hierarchical reinforcement learning. Neuron 71, 370–379 (2011). 10.1016/j.neuron.2011.05.042

[32]

Schlesiger, M.I. et al. The medial entorhinal cortex is necessary for temporal organization of hippocampal neuronal activity. Nat. Neurosci. 18, 1123–1132 (2015). 10.1038/nn.4056

[33]

Blum, K.I. & Abbott, L.F. A model of spatial map formation in the hippocampus of the rat. Neural Comput. 8, 85–93 (1996). 10.1162/neco.1996.8.1.85

[34]

Levy, W.B., Hocking, A.B. & Wu, X. Interpreting hippocampal function as recoding and forecasting. Neural Netw. 18, 1242–1264 (2005). 10.1016/j.neunet.2005.08.005

[35]

Hassabis, D. & Maguire, E.A. The construction system of the brain. Phil. Trans. R. Soc. Lond. B 364, 1263–1271 (2009). 10.1098/rstb.2008.0296

[36]

Buckner, R.L. The role of the hippocampus in prediction and imagination. Annu. Rev. Psychol. 61, 27–48, C1–C8 (2010). 10.1146/annurev.psych.60.110707.163508

[37]

Barry, C., Hayman, R., Burgess, N. & Jeffery, K.J. Experience-dependent rescaling of entorhinal grids. Nat. Neurosci. 10, 682–684 (2007). 10.1038/nn1905

[38]

Fanselow, M.S. From contextual fear to a dynamic view of memory systems. Trends Cogn. Sci. 14, 7–15 (2010). 10.1016/j.tics.2009.10.008

[39]

Wiltgen, B.J., Sanders, M.J., Anagnostaras, S.G., Sage, J.R. & Fanselow, M.S. Context fear learning in the absence of the hippocampus. J. Neurosci. 26, 5484–5491 (2006). 10.1523/jneurosci.2685-05.2006

[40]

Maurer, A.P. & McNaughton, B.L. Network and intrinsic cellular mechanisms underlying theta phase precession of hippocampal neurons. Trends Neurosci. 30, 325–333 (2007). 10.1016/j.tins.2007.05.002

[41]

Johnson, A. & Redish, A.D. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27, 12176–12189 (2007). 10.1523/jneurosci.3761-07.2007

[42]

Pezzulo, G., van der Meer, M.A., Lansink, C.S. & Pennartz, C.M. Internally generated sequences in learning and executing goal-directed behavior. Trends Cogn. Sci. 18, 647–657 (2014). 10.1016/j.tics.2014.06.011

[43]

Hasselmo, M.E. & Stern, C.E. Theta rhythm and the encoding and retrieval of space and time. Neuroimage 85, 656–666 (2014). 10.1016/j.neuroimage.2013.06.022

[44]

Ekstrom, A.D., Meltzer, J., McNaughton, B.L. & Barnes, C.A. NMDA receptor antagonism blocks experience-dependent expansion of hippocampal “place fields”. Neuron 31, 631–638 (2001). 10.1016/s0896-6273(01)00401-9

[45]

Hafting, T., Fyhn, M., Bonnevie, T., Moser, M.-B. & Moser, E.I. Hippocampus-independent phase precession in entorhinal grid cells. Nature 453, 1248–1252 (2008). 10.1038/nature06957

[46]

Dyna, an integrated architecture for learning, planning, and reacting

Richard S. Sutton

ACM SIGART Bulletin 1991 10.1145/122344.122377

[47]

Zhang, J., Springenberg, J.T., Boedecker, J. & Burgard, W. Deep reinforcement learning with successor features for navigation across similar environments. IEEE/RSJ International Conference on Intelligent Robots and Systems (2017). 10.1109/iros.2017.8206049

[48]

Momennejad, I. et al. The successor representation in human reinforcement learning. Preprint at https://doi.org/10.1101/083824 (2017). 10.1101/083824

[49]

Howard, M.W., Fotedar, M.S., Datey, A.V. & Hasselmo, M.E. The temporal context model in spatial navigation and relational learning: toward a common explanation of medial temporal lobe function across domains. Psychol. Rev. 112, 75–116 (2005). 10.1037/0033-295x.112.1.75

[50]

Krupic, J., Burgess, N. & O'Keefe, J. Neural representations of location composed of spatially periodic bands. Science 337, 853–857 (2012). 10.1126/science.1222403

Showing 50 of 55 references

Cited By

872

Hippocampal neuronal activity is aligned with action plans

Ipshita Zutshi, Athina Apostolelli · 2025

Nature

Local prediction-learning in high-dimensional spaces enables neural networks to plan

Christoph Stöckl, Yukun Yang · 2024

Nature Communications

Formation and retrieval of cell assemblies in a biologically realistic spiking neural network model of area CA3 in the mouse hippocampus

Jeffrey D. Kopsick, Joseph A. Kilgore · 2024

Journal of Computational Neuroscien...

Explaining dopamine through prediction errors and beyond

Samuel J. Gershman, John A. Assad · 2024

Nature Neuroscience

Representational spaces in orbitofrontal and ventromedial prefrontal cortex: task states, values, and beyond

Nir Moneta, Shany Grossman · 2024

Trends in Neurosciences

Toroidal topology of population activity in grid cells

Richard J. Gardner, Erik Hermansen · 2022

Nature

Navigating for reward

Marielena Sosa, Lisa M. Giocomo · 2021

Nature Reviews Neuroscience

The many dimensions of human hippocampal organization and (dys)function

Sarah Genon, Boris C. Bernhardt · 2021

Trends in Neurosciences

Psychoactive Effects of Lactobacillus johnsonii Against Restraint Stress-Induced Memory Dysfunction in Mice Through Modulating Intestinal Inflammation and permeability—a Study Based on the Gut–Brain Axis Hypothesis

Hesong Wang, Shunhui He · 2021

Frontiers in Pharmacology

Collective minds: social network topology shapes collective cognition

Ida Momennejad · 2021

Philosophical Transactions of the R...

Constant Sub-second Cycling between Representations of Possible Futures in the Hippocampus

Kenneth Kay, Jason E. Chung · 2020

Cell

Deep learning in business analytics and operations research: Models, applications and managerial implications

Mathias Kraus, Stefan Feuerriegel · 2020

European Journal of Operational Res...

Transition Scale-Spaces: A Computational Theory for the Discretized Entorhinal Cortex

Nicolai Waniek · 2020

Neural Computation

Navigating cognition: Spatial codes for human thinking

Jacob L. S. Bellmund, Peter Gärdenfors · 2018

Science

What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior

Timothy E.J. Behrens, Timothy H. Muller · 2018

Neuron

Metrics

872

Citations

55

References

Details

Published: Oct 02, 2017
Vol/Issue: 20(11)
Pages: 1643-1653
License: View

Authors

K

Kimberly L Stachenfeld

Cite This Article

Kimberly L Stachenfeld, Matthew M Botvinick, Samuel J Gershman (2017). The hippocampus as a predictive map. Nature Neuroscience, 20(11), 1643-1653. https://doi.org/10.1038/nn.4650

The hippocampus as a predictive map

You May Also Like