Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Rotational dynamics reduce interference between sensory and memory representations

Nature Neuroscience volume 24, pages 715–726 (2021)Cite this article

Subjects

Abstract

Cognition depends on integrating sensory percepts with the memory of recent stimuli. However, the distributed nature of neural coding can lead to interference between sensory and memory representations. Here, we show that the brain mitigates such interference by rotating sensory representations into orthogonal memory representations over time. To study how sensory inputs and memories are represented, we recorded neurons from the auditory cortex of mice as they implicitly learned sequences of sounds. We found that the neural population represented sensory inputs and the memory of recent stimuli in two orthogonal dimensions. The transformation of sensory information into a memory was facilitated by a combination of ‘stable’ neurons, which maintained their selectivity over time, and ‘switching’ neurons, which inverted their selectivity over time. Together, these neural responses rotated the population representation, transforming sensory inputs into memory. Theoretical modeling showed that this rotational dynamic is an efficient mechanism for generating orthogonal representations, thereby protecting memories from sensory interference.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Associative learning of sequences facilitates prediction.
Fig. 2: Sensory representations align with experience.
Fig. 3: Orthogonal memory representation avoids interference.
Fig. 4: Rotation creates an orthogonal memory representation.
Fig. 5: Rotation creates an orthogonal memory representation.
Fig. 6: Neurons with stable and switching selectivity rotate the sensory representation into a memory representation.
Fig. 7: Rotational dynamics is an efficient mechanism for generating orthogonality.

Similar content being viewed by others

Data availability

The data that support each main figure are included as source data. Original data are available upon reasonable request. Source data are provided with this paper.

Code availability

The code supporting the implementation and analysis of the neural network model is available on our lab GitHub repository (www.github.com/buschman-lab). As the model was analyzed in the same way as the neural data, the same analysis code can be applied to neural data.

References

  1. Summerfield, C. & de Lange, F. P. Expectation in perceptual decision making: neural and computational mechanisms. Nat. Rev. Neurosci. 15, 745–756 (2014).

    Article  CAS  PubMed  Google Scholar 

  2. Kiyonaga, A., Scimeca, J. M., Bliss, D. P. & Whitney, D. Serial dependence across perception, attention, and memory. Trends Cogn. Sci. 21, 493–497 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  3. de Lange, F. P., Heilbron, M. & Kok, P. How do expectations shape perception? Trends Cogn. Sci. 22, 764–779 (2018).

    Article  PubMed  Google Scholar 

  4. Fiser, A. et al. Experience-dependent spatial expectations in mouse visual cortex. Nat. Neurosci. 19, 1658–1664 (2016).

    Article  CAS  PubMed  Google Scholar 

  5. Jaramillo, S. & Zador, A. M. The auditory cortex mediates the perceptual effects of acoustic temporal expectation. Nat. Neurosci. 14, 246–251 (2011).

    Article  CAS  PubMed  Google Scholar 

  6. Chun, M. M. & Jiang, Y. Contextual cueing: implicit learning and memory of visual context guides spatial attention. Cogn. Psychol. 36, 28–71 (1998).

    Article  CAS  PubMed  Google Scholar 

  7. Dehaene, S., Meyniel, F., Wacongne, C., Wang, L. & Pallier, C. The neural representation of sequences: from transition probabilities to algebraic patterns and linguistic trees. Neuron 88, 2–19 (2015).

    Article  CAS  PubMed  Google Scholar 

  8. Carandini, M. & Heeger, D. J. Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13, 51–62 (2012).

    Article  CAS  Google Scholar 

  9. Buschman, T. J., Siegel, M., Roy, J. E. & Miller, E. K. Neural substrates of cognitive capacity limitations. Proc. Natl Acad. Sci. USA 108, 11252–11255 (2011).

    Article  CAS  PubMed  Google Scholar 

  10. Sprague, T. C., Ester, E. F. & Serences, J. T. Reconstructions of information in visual spatial working memory degrade with memory load. Curr. Biol. 24, 2174–2180 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Bouchacourt, F. & Buschman, T. J. A flexible model of working memory. Neuron 103, 147–160.e8 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. White, O. L., Lee, D. D. & Sompolinsky, H. Short-term memory in orthogonal neural networks. Phys. Rev. Lett. 92, 148102 (2004).

    Article  PubMed  CAS  Google Scholar 

  13. Botvinick, M. M. & Plaut, D. C. Short-term memory for serial order: a recurrent neural network model. Psychol. Rev. 113, 201–233 (2006).

    Article  PubMed  Google Scholar 

  14. Rigotti, M. et al. The importance of mixed selectivity in complex cognitive tasks. Nature 497, 585–590 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Sakai, K. & Miyashita, Y. Neural organization for the long-term memory of paired associates. Nature 354, 152–155 (1991).

    Article  CAS  PubMed  Google Scholar 

  16. Miyashita, Y. & Chang, H. S. Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature 331, 68–70 (1988).

    Article  CAS  PubMed  Google Scholar 

  17. Gavornik, J. P. & Bear, M. F. Learned spatiotemporal sequence recognition and prediction in primary visual cortex. Nat. Neurosci. 17, 732–737 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  18. Li, N. & DiCarlo, J. J. Unsupervised natural experience rapidly alters invariant object representation in visual cortex. Science 321, 1502–1507 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Maheu, M., Dehaene, S. & Meyniel, F. Brain signatures of a multiscale process of sequence learning in humans. eLife 8, e41541 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Kim, R., Seitz, A., Feenstra, H. & Shams, L. Testing assumptions of statistical learning: is it long-term and implicit? Neurosci. Lett. 461, 145–149 (2009).

    Article  CAS  PubMed  Google Scholar 

  21. Yakovlev, V., Fusi, S., Berman, E. & Zohary, E. Inter-trial neuronal activity in inferior temporal cortex: a putative vehicle to generate long-term visual associations. Nat. Neurosci. 1, 310–317 (1998).

    Article  CAS  PubMed  Google Scholar 

  22. Griniasty, M., Tsodyks, M. V. & Amit, D. J. Conversion of temporal correlations between stimuli to spatial correlations between attractors. Neural Comput. 5, 1–17 (1993).

    Article  Google Scholar 

  23. Amit, D., Brunel, N. & Tsodyks, M. Correlations of cortical Hebbian reverberations: theory versus experiment. J. Neurosci. 14, 6435–6445 (1994).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. den Ouden, H. E. M., Friston, K. J., Daw, N. D., McIntosh, A. R. & Stephan, K. E. A dual role for prediction error in associative learning. Cereb. Cortex 19, 1175–1185 (2009).

    Article  Google Scholar 

  25. Eagleman, D. M. Motion integration and postdiction in visual awareness. Science 287, 2036–2038 (2000).

    Article  CAS  PubMed  Google Scholar 

  26. Aru, J., Tulver, K. & Bachmann, T. It’s all in your head: expectations create illusory perception in a dual-task setup. Conscious. Cogn. 65, 197–208 (2018).

    Article  PubMed  Google Scholar 

  27. Choi, H. & Scholl, B. J. Perceiving causality after the fact: postdiction in the temporal dynamics of causal perception. Perception 35, 385–399 (2006).

    Article  PubMed  Google Scholar 

  28. Fischer, J. & Whitney, D. Serial dependence in visual perception. Nat. Neurosci. 17, 738–743 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Elsayed, G. F., Lara, A. H., Kaufman, M. T., Churchland, M. M. & Cunningham, J. P. Reorganization between preparatory and movement population responses in motor cortex. Nat. Commun. 7, 13239 (2016).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Itskov, P. M., Vinnik, E. & Diamond, M. E. Hippocampal representation of touch-guided behavior in rats: persistent and independent traces of stimulus and reward location. PLoS ONE 6, e16462 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Levine, J. H. et al. Data-driven phenotypic dissection of AML reveals progenitor-like cells that correlate with prognosis. Cell 162, 184–197 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Rigotti, M., Ben Dayan Rubin, D. D., Wang, X.-J. & Fusi, S. Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses. Front. Comput. Neurosci. 4, 24 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  33. Olshausen, B. A. & Field, D. J. Sparse coding of sensory inputs. Curr. Opin. Neurobiol. 14, 481–487 (2004).

    Article  CAS  PubMed  Google Scholar 

  34. Rust, N. C. & DiCarlo, J. J. Balanced increases in selectivity and tolerance produce constant sparseness along the ventral visual stream. J. Neurosci. 32, 10170–10182 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Bassett, D. S. & Sporns, O. Network neuroscience. Nat. Neurosci. 20, 353–364 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Murray, J. D. et al. Stable population coding for working memory coexists with heterogeneous neural dynamics in prefrontal cortex. Proc. Natl Acad. Sci. USA 114, 394–399 (2017).

    Article  CAS  PubMed  Google Scholar 

  37. Stokes, M. G. et al. Dynamic coding for cognitive control in prefrontal cortex. Neuron 78, 364–375 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Freedman, D. J., Riesenhuber, M., Poggio, T. & Miller, E. K. Visual categorization and the primate prefrontal cortex: neurophysiology and behavior. J. Neurophysiol. 88, 929–941 (2002).

    Article  PubMed  Google Scholar 

  39. Fuster, J. M. & Alexander, G. E. Neuron activity related to short-term memory. Science 173, 652–654 (1971).

    Article  CAS  PubMed  Google Scholar 

  40. Warden, M. R. & Miller, E. K. The representation of multiple objects in prefrontal neuronal delay activity. Cereb. Cortex 17, i41–i50 (2007).

    Article  PubMed  Google Scholar 

  41. Spaak, E., Watanabe, K., Funahashi, S. & Stokes, M. G. Stable and dynamic coding for working memory in primate prefrontal cortex. J. Neurosci. 37, 6503–6516 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Miller, P. & Wang, X.-J. Inhibitory control by an integral feedback signal in prefrontal cortex: a model of discrimination between sequential stimuli. Proc. Natl Acad. Sci. USA 103, 201–206 (2006).

    Article  CAS  PubMed  Google Scholar 

  43. Postle, B. R. The cognitive neuroscience of visual short-term memory. Curr. Opin. Behav. Sci. 1, 40–46 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  44. Chaudhuri, R. & Fiete, I. Computational principles of memory. Nat. Neurosci. 19, 394–403 (2016).

    Article  CAS  PubMed  Google Scholar 

  45. Meyers, E. M. Dynamic population coding and its relationship to working memory. J. Neurophysiol. 120, 2260–2268 (2018).

    Article  PubMed  Google Scholar 

  46. Riley, M. R. & Constantinidis, C. Role of prefrontal persistent activity in working memory. Front. Syst. Neurosci. 9, 181 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Perez, F. & Granger, B. E. IPython: a system for interactive scientific computing. Comput. Sci. Eng. 9, 21–29 (2007).

    Article  CAS  Google Scholar 

  48. Millman, K. J. & Aivazis, M. Python for scientists and engineers. Comput. Sci. Eng. 13, 9–12 (2011).

    Article  Google Scholar 

  49. Oliphant, T. E. Python for scientific computing. Comput. Sci. Eng. 9, 10–20 (2007).

    Article  CAS  Google Scholar 

  50. Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).

    Google Scholar 

  51. Walt, S., van der Colbert, S. C. & Varoquaux, G. The NumPy Array: a structure for efficient numerical computation. Comput. Sci. Eng. 13, 22–30 (2011).

    Article  Google Scholar 

  52. McKinney, W. Data structures for statistical computing in Python. Proc. 9th Python Sci. Conf. 445, 56–61 (2010).

    Article  Google Scholar 

  53. Hunter, J. D. Matplotlib: a 2D graphics environment. Comput. Sci. Eng. 9, 90–95 (2007).

    Article  Google Scholar 

  54. Manly, B. Randomization, Bootstrap and Monte Carlo Methods in Biology (Chapman & Hall/CRC, 1997).

  55. Nicosia, V., Mangioni, G., Carchiolo, V. & Malgeri, M. Extending the definition of modularity to directed graphs with overlapping communities. J. Stat. Mech. Theory Exp. 2009, P03024 (2009).

    Article  CAS  Google Scholar 

  56. Blondel, V. D., Guillaume, J.-L., Lambiotte, R. & Lefebvre, E. Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008, P10008 (2008).

    Article  Google Scholar 

  57. Rousseeuw, P. J. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987).

    Article  Google Scholar 

  58. McInnes, L., Healy, J. & Melville, J. UMAP: uniform manifold approximation and projection for dimension reduction. Preprint at arXiv https://arxiv.org/abs/1802.03426 (2018).

  59. Beyer, K., Goldstein, J., Ramakrishnan, R. & Shaft, U. in Database Theory—ICDT’99 (eds Beeri, C. & Buneman, P.) 217–235 (Springer, 1999).

  60. Wasmuht, D. F., Spaak, E., Buschman, T. J., Miller, E. K. & Stokes, M. G. Intrinsic neuronal dynamics predict distinct functional roles during working memory. Nat. Commun. 9, 3499 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank C. MacDowell, M. Panichello, C. Jahn, F. Bouchacourt, P. Hoyos and S. Henrickson for their detailed feedback during the writing of this manuscript. We also thank B. Briones for helping with histology and B. Morea for helping with surgery. We thank the Princeton Laboratory Animal Resources staff for their support. This work was supported by NIMH R01MH115042, ONR N000141410681 and NIH DP2EY025446 to T.J.B.

Author information

Authors and Affiliations

  1. Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA

    Alexandra Libby & Timothy J. Buschman

  2. Department of Psychology, Princeton University, Princeton, NJ, USA

    Timothy J. Buschman

Authors
  1. Alexandra Libby
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy J. Buschman
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

T.J.B. and A.L. conceived the project and designed the experiments. A.L. did surgery on the animals, collected the data, constructed computational models and analyzed the data, with supervision from T.J.B. A.L. and T.J.B. wrote the paper.

Corresponding author

Correspondence to Timothy J. Buschman.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Neuroscience thanks Omri Barak and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Encoding along the B/Y Sensory Axis.

a, The neural population encoding of B/Y shown on (a) Day 1 and (b) Day 4. For each of the four conditions, the plot shows the mean ± s.e.m. of the population projection onto the B/Y sensory axis. Yellow outlines B/Y training period (185–285 ms). For panels a-e, n = 1064 withheld trials, z-scored and then combined across animals per day. Positive and negative projections indicate Y (green) and B (purple) encoding, respectively. Light and dark grey horizontal bars mark significant differences for AB vs XY and C vs. C*, respectively (two-sided t-tests, p ≤ 0.001, Bonferroni corrected). c, Data show mean ± s.e.m. of B/Y stimulus encoding strength on the B/Y sensory axis. Negatively labeled conditions (that is, B) were inverted, such that positive values on y-axis indicate B and Y trials are ‘correctly’ encoded as B and Y, respectively. Day 1 = 0.34 ± 0.021, Day 2 = 0.33 ± 0.22, Day 3 = 0.33 ± 0.022, Day 4 = 0.31 ± 0.021, all days p < 1/5000 two-sided bootstrap tests. Slope across days mean ± s.e.m. = −0.01 ± 0.01, p = 0.16, one-sided bootstrap test. d, Points show mean ± s.e.m. of A/X stimulus encoding strength on the B/Y sensory axis, during A/X stimulus presentation. For panels d-f, lines and shaded regions show mean and 95% CI of bootstrapped linear regressions. Positive values indicate correct A/X encoding: Day 1 = 0.13 ± 0.021, p < 1/5000, Day 2 = 0.062 ± 0.023, p = 0.0064, Day 3 = 0.096 ± 0.022, Day 4 = 0.031 ± 0.023, p = 0.17, all two-sided bootstrap tests. Slope across days mean ± s.e.m. = −0.028 ± 0.01, p = 0.0016, one-sided bootstrap test. e, Points show mean ± s.e.m. of C/C* stimulus encoding strength on the B/Y sensory axis. Positive values indicate correct encoding of C/C* association on the B/Y sensory axis (that is, C and C* should go in B and Y direction, respectively). Day 1 = −0.10 ± 0.023, p < 1/5000, Day 2 = −0.056 ± 0.024, p = 0.016, Day 3 = −0.006 ± 0.023, p = 0.81, Day 4 = −0.044 ± 0.023, p = 0.055, all two-sided bootstrap tests. Slope across days mean ± s.e.m. = 0.022 ± 0.01, p = 0.017, one-sided bootstrap test. Note, this trend does not appear in analysis of blocks of trials within a day (Supplementary Fig. 2f). f-g, Points show mean ± s.e.m. of angles between B/Y sensory axis and (f) A/X and (g) C/C* sensory axes (n = 5000 resamples of neurons). Significant differences from 90 degrees shown by grey boxes (p≤0.01, one-sided bootstrap tests). Significant change in angle to A/X sensory axis over time is shown by grey line (shaded region is 95% confidence interval of bootstrapped linear regression). The B/Y and A/X sensory axes were initially aligned, but became orthogonal over days: change over days, slope = 0.84 ± 0.24, p < 1/5000, one-sided bootstrap test. B/Y and C/C* sensory axes were always orthogonal. Change over days: slope = 0.29 ± 0.2, p = 0.077, one-sided bootstrap test. For all panels, p-values: * ≤ 0.05, ** ≤ 0.01, *** ≤ 0.001.

Extended Data Fig. 2 Classifier Performance over Sequence Timecourse.

For each classifier, the accuracy (y-axis) was measured as the area under the curve (AUC; see methods). Accuracy was calculated using data from trials withheld from training (n = 152 trials per animal) and was calculated in a sliding window fashion (25 ms windows, stepped 10 ms). Lines show mean ± s.e.m. of accuracy timecourses (n = 7 animals). Day 1 and 4 shown in left and right panels, respectively. a, A/X sensory classifier performance over time, shown for decoding A/X (orange) and C/C* (blue) stimuli. Orange rectangle indicates A/X training period (10–110 ms). b, C/C* sensory classifier performance over time. Line colors follow panel a. Blue rectangle indicates C/C* training period (360–460 ms). Consistent with predictive coding shown by projections in Fig. 1g-h, on Day 4 the C/C* sensory classifier decoded A/X during the A/X stimulus and immediately before C/C*. c, A/X sensory classifier performance shown for expected trials (black line - ABCD vs. XYC*D) and unexpected trials (grey line - ABC*D vs. XYCD). Consistent with postdiction results shown in Fig. 3b, the A/X sensory classifier performs well on expected trials, but incorrectly classifies A/X during unexpected C/C* trials. d, A/X memory classifier performance over time, shown for A/X discrimination (dark blue) and C/C* (light blue). Dark blue rectangle indicates training period (360–460 ms). Consistent with projection results shown in Fig. 3c-d and Extended Data Figs. 5 and 6, A/X memory classifier can decode A/X near the end, but not beginning of the trial. A/X discrimination of C/C* is close to chance (AUC = 0.5), reflecting the fact that the two axes are independent. e, A/X memory classifier performance, divided by expectation. Colors follow panel c. The A/X memory classifier performs well at discriminating A/X on both expected and unexpected trials.

Extended Data Fig. 3 Associative Learning Generalizes to C/C* Chords Presented Outside of Sequence.

a-b, Lines show mean ± s.e.m. of neural activity (trials balanced across conditions) projected onto a C/C* chord encoding axis on (a) Day 1 (n = 4204) and (b) Day 4 (n = 4196). The C/C* chord encoding axis was trained using the firing rate response to the C/C* chord presented in isolation, outside of sequences (n = 300 trials). Line colors indicate trial types (ABCD – orange, ABC*D – pink, XYCD – green, XYC*D – blue) and line style indicates 3rd chord type (C – solid, C* – dashed). Positive and negative projections indicate C* and C encoding, respectively. Light and dark grey horizontal bars mark significant differences for AB vs. XY and C vs. C*, respectively (two-sided t-test, p ≤ 0.001, Bonferroni corrected). Results are consistent with projections onto the C/C* sensory axis (Fig. 1g,h). c, Points show mean ± s.e.m. of C/C* prediction strength during the A/X stimulus, which grew over days. Positive prediction (y-axis) indicates the C/C* chord sensory axis correctly encoded the association (that is, C during A and C* during X) during the A/X stimulus (black outline in panels a-b, 10–110 ms). Day 1 = 0.017 ± 0.023, p = 0.44, Day 2 = 0.025 ± 0.023, p = 0.27, Day 3 = 0.071 ± 0.023, p = 0.0008, Day 4 = 0.14 ± 0.022, p < 1/5000, two-sided bootstrap tests. Trials used in other projection analyses were also used here (n = 1064). For panels c-d, lines and shaded region show mean and 95% CI of bootstrapped linear regressions. Consistent with Fig. 1i, the prediction along the C/C* sensory chord axis increased across days; slope mean ± s.e.m. = 0.04 ± 0.01, p < 1/5000, one-sided bootstrap test. d, Violin plots show bootstrapped distributions of the angle between A/X sensory and C/C* chord sensory axes (n = 5000 resamples of neurons). The mean ± s.e.m. angle between axes by day (degrees): Day 1 = 95 ± 6.5, p = 0.19; Day 2 = 84 ± 5.9, p = 0.16, Day 3 = 78 ± 5.0, p = 0.011, Day 4 = 83 ± 4.4, p = 0.064, one-sided bootstrap tests against 90 degrees. Regression across days: slope = −4.3 ± 2.5, p = 0.039, one-sided bootstrap test. e, Angle between A/X memory and C/C* chord sensory axes. Angle (degrees) on Day 1 = 88 ± 7.0, p = 0.36; Day 2 = 103 ± 6.3, p = 0.019, Day 3 = 95 ± 4.5, p = 0.14, Day 4 = 83 ± 5.2, p = 0.10, one-sided bootstrap tests against 90 degrees. Regression across days: slope = −2.2 ± 2.8, p = 0.21, one-sided bootstrap test. For all panels, p-values: * ≤ 0.05, ** ≤ 0.01, *** ≤ 0.001.

Extended Data Fig. 4 Alignment of Neural Activity in A/X-C/C* State Space.

a, Neural activity projected into A/X-C/C* state space for Day 1 (left) and Day 4 (right). Lines show mean projections of neural activity onto the A/X sensory axis (x-axis) and C/C* sensory axis (y-axis; n = 1064 trials). Activity is shown during the A/X stimulus presentation (−10–170 ms) for each of four trial types, indicated by legend. Marker saturation increases with time, as shown in sequence timecourse legend above graph. Inset shows principal components (PCs) of neural trajectories in grey; black arrow size matches percentage of explained variance per PC (see methods). On day 1, the neural trajectory moved predominately along the A/X encoding axis (x-axis). By day 4, the neural trajectories followed an angle, encoding both A/X and the expected C/C* information (y-axis). b, The angle of PC1 (relative to horizontal) during the A/X period increased across days. Radial lines show the circular mean ± s.e.m. of angle shown for Day 1 (light grey) and Day 4 (dark grey). Angle of PC1 per day (degrees): Day 1 = 18 ± 2.9, Day 2 = 14 ± 3.6, Day 3 = 11 ± 4.7, Day 4 = 31 ± 2.3 degrees (bootstrap, n = 5000 resamples of neurons). Change in angle across days, slope = 3.7 ± 2.4, p = 0.0028, one-sided bootstrap test. c, Neural activity during the C/C* stimulus period (340 to 520 ms) projected into A/X-C/C* state space, as in panel a. d, The angle of PC1 (relative to vertical) during the C/C* period decreased across days. Format follows panel b. Angle of PC1 per day (degrees): Day 1 = 79 ± 3.5, Day 2 = 74 ± 3.8, Day 3 = 77 ± 4.3, Day 4 = 58 ± 2.7 (bootstrap); change in angle across days, slope = −6.0 ± 1.4, p < 1/5000, one-sided bootstrap test. For all panels, p-values: * ≤ 0.05, ** ≤ 0.01, *** ≤ 0.001.

Extended Data Fig. 5 A/X Memory Representation and Full Neural Dimensionality.

a,b, The neural population encoding of A/X memory shown on (a) Day 1 and (b) Day 4. For each of the four conditions, the lines show the mean ± s.e.m. of the population projection onto the A/X memory axis (blue outlines A/X memory training period; for all panels, n = 1064 withheld trials, combined across animals per day). Positive and negative projections indicate XY and AB encoding, respectively. Light and dark grey bars mark significant differences for AB vs. XY and C vs. C* respectively (two-sided t-test, p≤0.001, Bonferroni corrected). c-d, Neural activity projected into A/X memory - C/C* state space for (c) Day 1 and (d) Day 4, around the C/C* stimulus presentation (340–520 ms). The x-axis and y-axis are the projections of neural activity onto the A/X memory axis and the C/C* sensory axis, respectively. Marker saturation increases with time (shown across top). Inset shows PCs of neural trajectories in grey; black arrow size matches percentage of explained variance per PC (for distributions see Fig. 3h). e, Violin plots show distribution of the dimensionality of the full neural response during the C/C* stimulus presentation. For each day, PCA was performed on the firing rate responses across a pseudo population (neurons were concatenated across animals; see methods). Similar to Fig. 3h, the dimensionality was estimated using the explained variance ratio (EVR) of the first two PCs (see methods). Dimensionality of the neural responses tended to decrease over days, as shown by the increased in the EVR of first two PCs: Day 1 = 0.63 ± 0.062, Day 2 = 0.54 ± 0.056, Day 3 = 0.68 ± 0.064, Day 4 = 0.76 ± 0.091 (bootstrap, n = 5000 resamples of neurons). Change in EVR of first two PCs over days: slope mean ± s.e.m. = 0.053 ± 0.034, p = 0.065, one-sided bootstrap test.

Extended Data Fig. 6 A/X Sensory to Memory Transformation.

a, Cross-temporal performance of A/X classifiers. A series of A/X classifiers were trained across the sequence (x-axis; 25 ms windows, stepping by 10 ms) and then each classifier was tested across the sequence (y-axis). Color indicates the average correct projection on withheld data for all combinations of training times and test times. White bars indicated onset and offset of A/X, B/Y, C/C* stimuli. Note, the low cross-temporal decoding performance between the A/X and C/C* time periods reflects the temporal dynamics of the representation of A/X during the sequence. b, Points show mean ± s.e.m. of correct projection along the A/X sensory axis (orange) and A/X memory axis (blue), during the first three stimuli in the sequence (A/X, B/Y, and C/C* columns). Positive values indicate correct encoding strength; negatively encoded conditions (that is, A) were inverted before averaging. Horizontal bars indicate significant differences between A/X encoding during the A/X stimulus and C/C* stimulus. The A/X sensory axis had stronger A/X encoding during A/X sensory compared to the C/C* stimulus (differences per day: Day 1 = 0.31, Day 2 = 0.26, Day 3 = 0.27, Day 4 = 0.32, all p≤1/5000, one-sided permutation tests). The A/X memory axis had stronger encoded A/X encoding during the C/C* stimulus compared to the A/X stimulus (differences per day: Day 1 = −0.19, Day 2 = −0.22, Day 3 = −0.11, Day 4 = −0.17, all p = 0.0002, one-sided permutation tests). (A/X Stimulus) Projections of neural activity during the A/X stimulus (10–110 ms) onto A/X sensory axis (mean ± s.e.m.): Day 1 = 0.37 ± 0.2, Day 2 = 0.27 ± 0.022, Day 3 = 0.28 ± 0.022, Day 4 = 0.33 ± 0.021, all p < 1/5000. Onto A/X memory axis: Day 1 = 0.053 ± 0.023, p = 0.022, Day 2 = 0.06 ± 0.024, p = 0.013, Day 3 = 0.12 ± 0.023, p < 1/5000, Day 4 = 0.053 ± 0.023, p = 0.019 (all two-sided bootstrap tests). During the A/X stimulus, A/X sensory encoding was stronger than A/X memory encoding on all days (Sen. – Mem. differences: Day 1 = 0.31, Day 2 = 0.21, Day 3 = 0.16, Day 4 = 0.27, all p≤1/5000, one-sided permutation tests). (B/Y Stimulus) Projections of neural activity during the B/Y stimulus (180–280 ms) onto the A/X sensory axis: Day 1 = 0.12 ± 0.022, p < 1/5000, Day 2 = 0.0038 ± 0.024, p = 0.87, Day 3 = 0.12 ± 0.023, p < 1/5000, Day 4 = 0.046 ± 0.024, p = 0.046. Onto A/X memory axis: Day 1 = 0.07 ± 0.023, p = 0.0044, Day 2 = 0.22 ± 0.23, p < 1/5000, Day 3 = 0.19 ± 0.23, p < 1/5000, Day 4 = 0.11 ± 0.024, p < 1/5000 (all two-sided bootstrap tests). During B/Y stimulus, A/X sensory encoding was slightly stronger than A/X memory on Day 1 (Sen. – Mem. diff. = 0.05, p = 0.064), but after experience, A/X memory encoding of A/X information was significantly stronger than A/X sensory encoding (Day 2 = −0.21, p = 0.0002, Day 3 = −0.07, p = 0.017, Day 4 diff. = −0.07, p = 0.02, all one-sided permutation tests). (C/C* Stimulus) Projections of neural activity during the C/C* stimulus (360–460 ms) onto A/X sensory axis: Day 1 = 0.053 ± 0.023, p = 0.021, Day 2 = 0.0034 ± 0.023, p = 0.87, Day 3 = 0.008 ± 0.023, p = 0.72, Day 4 = 0.0069 ± 0.023, p = 0.77. Onto A/X memory axis: Day 1 = 0.24 ± 0.023, Day 2 = 0.28 ± 0.023, Day 1 = 0.24 ± 0.024, Day 4 = 0.23 ± 0.024, all p < 1/5000 (all two-sided bootstrap tests). During the C/C* stimulus, the A/X memory encoding was stronger than A/X sensory encoding on all days (Sen – Mem. differences: Day 1 = −0.19, Day 2 = −0.28, Day 3 = −0.23, Day 4 = −0.22, p = 0.0002, all one-sided permutation tests). c, Violin plots show distribution of when A/X memory encoding strength crossed A/X sensory encoding strength. Horizontal line indicates mean. Mean ± s.e.m. of switch times (ms) relative to sequence onset: Day 1 = 248 ± 13, Day 2 = 182 ± 10, Day 3 = 178 ± 22, Day 4 = 194 ± 22 (n = 5000, bootstrap over trials). The switch time decreased over days (slope mean ± s.e.m. = −16 ± 8.2, p = 0.02, one-sided bootstrap test). The change in switch time decreased the most between days 1 and 3 and then stabilized by day 4 (Day 4-3 diff. = 16.11 ± 31, p = 0.31, one-sided bootstrap test). For all panels, p-values: * ≤ 0.05, ** ≤ 0.01, *** ≤ 0.001.

Extended Data Fig. 7 A/X Sensory and A/X Memory Encoding have Opposite Effects on C/C* Encoding.

Trial-by-trial correlation of encoding strength along three relevant axes: A/X sensory, C/C* sensory, and A/X memory. Positive and negative values on each encoding axis indicate correct and incorrect projections, respectively. All lines show mean and 95% confidence interval of bootstrapped linear regressions; slope, correlation (r) and p-values (all one-sided bootstrap tests, uncorrected for multiple comparisons across panels) are listed in plots. a-b, Correlation between A/X sensory encoding strength (x-axis; 10–110 ms) and A/X memory encoding strength (y-axis; 360–460 ms) on (a) Day 1 and (b) Day 4. Consistent with a transformation of A/X information from sensory to memory, there is a significant correlation on Day 1 and 4. c-f, Relationship between A/X encoding strength (x-axis) and C/C* sensory encoding strength (y-axis). A/X encoding strength by the sensory and memory axes was estimated during the 50 ms prior to C/C* onset (300–350 ms). C/C* sensory encoding strength was estimated during C/C* (360–460 ms). Panels show correlations between C/C* representation and A/X sensory representation (c and d) or A/X memory representation (e and f). Correlations are shown for both expected stimuli (c and e; ABCD, XYC*D) and unexpected stimuli (d and f; ABC*D, XYCD). c, On day 4, A/X sensory encoding was positively correlated with C/C* encoding accuracy on expected trials (ABCD, XYC*D). d, On day 4, A/X sensory encoding was negatively correlated with C/C* encoding accuracy on unexpected trials (ABC*D, XYCD). e, On day 4 there was no significant correlation between A/X memory encoding accuracy and C/C* encoding on expected trials. f, On day 4, A/X memory encoding accuracy was positively correlated with C/C* encoding during unexpected trials.

Extended Data Fig. 8 Dynamics of A/X Selectivity Are Consistent across C/C* Stimuli.

Phenograph clustering (Fig. 5a) was applied to z-scored firing rate differences calculated for specific C/C*/Cmix stimuli. a, Z-scored differences were calculated for XYCD-ABCD (left), XYC*D-ABC*D (middle), and XYCmixD-ABCmixD (right) pairs of conditions. Lines show mean ± s.e.m. of A/X selectivity over time per original Phenograph cluster (n = 522). Note, Cmix trials involved presenting a novel stimulus that was a mix between the two chords making up C and C*; ABCmixD and XYCmixD sequences occurred on 12% of trials, randomly distributed throughout the day (see methods). b, Data points show the individual neurons’ original AB-XY z-scored differences (x-axis) were highly correlated with z-scored differences calculated on the ‘C’ trials (XYCD-ABCD), during sensory (dark blue, left; r = 0.91 ± 0.01) and memory (dark blue, right; r = 0.89 ± 0.02) time periods. Similarly, the correlation was high to ‘C*’ trials (XYCD-ABC*D) during sensory (blue, left; r = 0.92 ± 0.01) and memory (blue, right; r = 0.87 ± 0.02) time periods. Finally, this correlation was also seen on Cmix trials (XYCmixD-ABCmixD) during both sensory (green, left; r = 0.8 ± 0.02) and memory (green, right; r = 0.73 ± 0.02) time periods. All correlations were significant (p < 1/5000, one-sided bootstrapped linear regressions, n = 5000 resamples across neurons).

Extended Data Fig. 9 Stable and Switching Dynamics Capture the Temporal Dynamics of Single Neurons.

a, Sensitivity index (d-prime) calculated between all pairs of the four Phenograph clusters (see methods). Red line shows observed d-prime; histograms show d-prime after permutation (1000 shuffles). All clusters were more separated than expected by chance (all p≤0.001, one-sided permutation tests). b, Plot shows how systematically varying the number of neighbors in the Phenograph algorithm (K; color-axis) changed the goodness of clustering, as measured by the silhouette score (x-axis) and modularity (y-axis, see methods). White text shows the resulting number of identified clusters. A K value between 35 and 45 results in 4 clusters and high silhouette scores and modularity. Increasing the K value beyond this recommended range leads to unstable clustering with highly variable silhouette scores and low modularity. c, Density of UMAP projection of A/X temporal selectivity. Dot colors indicate Phenograph clustering (left) and K-means clustering (right, number of clusters = 4) labels. Area of circle indicates number of data points in region (max size = 8). d, K-means silhouette score as a function of cluster number. K-means was performed on UMAP projections for timecourse of A/X selectivity, random selectivity, and C/C* selectivity. e-g, A/X temporal selectivity profile clustered by K-means applied to UMAP, as shown in panel c. Lines show mean ± s.e.m. of each cluster’s selectivity timecourse, after each K-means run, when the number of clusters set to (e) k = 2, (f) k = 3, and (g) k = 4.

Extended Data Fig. 10 Properties of Stable and Switching Neurons.

a, Intrinsic variability was higher in stable neurons. Lines show mean ± s.e.m. of fano factor of stable (orange, n = 355) and switching (blue, n = 167) neurons over the sequence (neurons combined days). b, Violin plots show distribution of fano factor during pre-stimulus period (−400–0 ms), stimulus presentation (A/X, B/Y, C/C* and D/D* combined, 100 ms each) and inter-chord interval (ICI; 75 ms each). Fano factor was higher in stable neurons compared to switching neurons before the stimulus (stable, mean ± s.e.m. = 1.06 ± 0.01, switching = 1.04 ± 0.01; diff. = −0.02, p = 0.02), during stimulus presentation (stable = 1.03 ± 0.01, switching = 1.01 ± 0.01; diff. = −0.02, p = 0.016), and during the ICI (stable = 1.03 ± 0.01, switching = 1.01 ± 0.01; diff. = −0.02, p = 0.01, all one-sided permutation tests). Difference between the pre-stimulus and stimulus periods were significant for both neuron types (stable = 0.03, p = 0.001; switching = 0.03, p = 0.03; one-sided permutation tests). c, Line show mean ± s.e.m. of intrinsic autocorrelation of functional neuron types, calculated during the pre-stimulus period. The autocorrelation at lag zero was removed for clarity. d, Histograms show distribution of time constants from autocorrelations. The time constant (tau; x-axis) provides a measure of each neuron types’s intrinsic timescale; it was estimated by fitting an exponential function to the autocorrelation shown in panel c. No difference was observed between neuron types: switching mean ± s.e.m. = 94 ± 51 ms, stable = 86 ± 31 ms (bootstrapped exponential fit; n = 1000 resamples with replacement). e, Switching neurons carried slightly more of the A/X-C/C* association than stable neurons. Lines show mean ± s.e.m. of stable (orange) and switching (blue) neurons’ A/X and C/C* temporal selectivity profiles. Neurons without significant C/C* selectivity were removed (stable n = 123; switching, n = 24, data combined across days). Selectivity of neurons is plotted with respect to their initial A/X preference (that is, initial selectivity is always positive). Dashed lines show C/C* selectivity of the same neurons. Responses to associated stimuli (AC/XC*) are positive, while responses to unassociated stimuli (AC*/XC) are negative. f, Violin plots show distribution of average predictive selectivity during C/C* stimulus presentation (350–450 ms). Each dot is a neuron; all days included. The mean ± s.e.m. of prediction in stable neurons = −0.13 ± 0.31, p = 0.67; switching neurons = 1.16 ± 0.53, p = 0.022, two-sided bootstrap tests. g, Data points show estimated locations of neurons in each functional cluster along recording array. Switching (blue), stable (orange), and none (grey) neurons are plotted according to their estimated electrode location (x-axis – AP, y-axis – depth (DV) based on implant coordinates; 6 probes had 4 shanks separated by 200 µm). Small, random jitter in anterior-posterior (AP) direction was added for clarity of presentation and does not reflect actual differences. For all panels, p-values: * ≤ 0.05, ** ≤ 0.01, *** ≤ 0.001.

Supplementary information

Supplementary Information

Supplementary Tables 1–4 and Supplementary Figs. 1–7.

Reporting Summary

Source data

Source Data Fig. 1

Statistical source data.

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Source Data Fig. 5

Statistical source data.

Source Data Fig. 6

Statistical source data.

Source Data Fig. 7

Statistical source data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Libby, A., Buschman, T.J. Rotational dynamics reduce interference between sensory and memory representations. Nat Neurosci 24, 715–726 (2021). https://doi.org/10.1038/s41593-021-00821-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41593-021-00821-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing