Neuromodulation (the localized manipulation of neuronal activity) holds therapeutic promise for neurological/neuropsychiatric diseases. There is currently an unmet need for a non-invasive, safe, personalized and inexpensive method of neuromodulation that can be used to functionally target and manipulate brain networks to achieve therapeutic goals. The lab’s long-term goal is to develop a novel neuromodulatory method based on individually designed sensory stimulation. While there have been notable successes in characterizing the “forward” mapping from sensory stimulus to brain response, harnessing this strategy for neuromodulation requires a solution to the inverse problem: designing sensory stimuli that activate pre-specified targets. Using the visual system as a model, the lab proposes building a computational procedure that, given a targeted brain region or network to activate, can synthesize personalized, 2D images that achieve that target. The lab’s central hypothesis is that combining a generative artificial neural network with information about an individual’s brain will enable such a procedure. The lab will create 1) a computational model that can predict brain responses to 2D images and 2) synthetic images that can achieve a predetermined target brain response. The lab will validate the approach via functional MRI (fMRI) experiments in 20 individuals. The next step: to show repeated presentation of the synthetic stimuli could predictably change the brain’s connectivity network, and show that these changes could alleviate symptoms or boost recovery. The project is significant in that it could be used to develop a novel, non-invasive, inexpensive, and convenient neuromodulatory tool that could lessen the burden of neuropsychiatric disease.
The lab has begun performing prospective experiments, and recently published the synthetic image generation framework (NeuroGen). NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. The lab demonstrates that the reduction of noise the encoding model provides, coupled with the generative network’s ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. Using only a small number of synthetic images created by NeuroGen, the lab demonstrates that it can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. The lab then verifies these discoveries are reflected in the several thousand observed image responses measured with fMRI. The lab further demonstrates NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system. Gu, Z., Jamison, K.W., Khosla M., Allen, E.J., Wu, Y., St-Yves, G., Naselaris, T., Kay, K., Sabuncu, M.R., Kuceyeski A. (2022) NeuroGen: Activation optimized image synthesis for discovery neuroscience. NeuroImage. (247):118812.
Multi-region optimization can create synthetic images with more extreme predicted activation than natural images. Scatter plots published by the group in the above NeuroImage publication show predicted response from top 10 natural and synthetic images that either jointly maximize the sum of the two regions (+ROI1+ROI2) or jointly maximize one and minimize the other (+ROI1- ROI2) and (-ROI1+ROI2), for the region-pairs FFA1 and V1v; PPA and V1v; and FFA1 and PPA. All sets of three-region optimization combinations for FFA1, PPA and V1v appear in this image from that publication. Typical synthetic image examples are shown for each dual and triple optimization; optimization specifics are below each synthetic image (green=that region’s minimization, red=that region’s maximization).