Title | Cortical response to naturalistic stimuli is largely predictable with deep neural networks. |
Publication Type | Journal Article |
Year of Publication | 2021 |
Authors | Khosla M, Ngo GH, Jamison K, Kuceyeski A, Sabuncu MR |
Journal | Sci Adv |
Volume | 7 |
Issue | 22 |
Date Published | 2021 05 |
ISSN | 2375-2548 |
Abstract | Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict neural responses to arbitrary stimuli can be very useful for studying brain function. However, existing models focus on limited aspects of naturalistic stimuli, ignoring the dynamic interactions of modalities in this inherently context-rich paradigm. Using movie-watching data from the Human Connectome Project, we build group-level models of neural activity that incorporate several inductive biases about neural information processing, including hierarchical processing, temporal assimilation, and auditory-visual interactions. We demonstrate how incorporating these biases leads to remarkable prediction performance across large areas of the cortex, beyond the sensory-specific cortices into multisensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize to task-bound paradigms. Together, our findings underscore the potential of encoding models as powerful tools for studying brain function in ecologically valid conditions. |
DOI | 10.1126/sciadv.abe7547 |
Alternate Journal | Sci Adv |
PubMed ID | 34049888 |
PubMed Central ID | PMC8163078 |
Grant List | R01 AG053949 / AG / NIA NIH HHS / United States R01 LM012719 / LM / NLM NIH HHS / United States R21 NS104634 / NS / NINDS NIH HHS / United States R01 NS102646 / NS / NINDS NIH HHS / United States |
Related Institute:
Brain Health Imaging Institute (BHII)