Hierarchies of spatiotemporal anticipation in the human brain

PROJECT SUMMARY Everyday experience requires humans to make plans in a hierarchical fashion, so they can anticipate visual events that might occur seconds, minutes, hours, or longer in the future. For example, when walking through a city, individuals must track their immediate surroundings (e.g., the movement of pedestrians around them), intermediate sub-goals (e.g., landmarks along their route), and long-timescale goals (e.g., their final destination). Despite the ubiquity of such hierarchical anticipation in behavior, it is unclear how the brain can simultaneously anticipate events at multiple timescales. Our aim is to uncover the mechanisms underlying hierarchical anticipatory signals in the brain’s visual system, by determining how these signals form, what they represent, how they are updated, and how they guide future-oriented behavior. This will be accomplished with functional magnetic resonance imaging (fMRI), neuropsychological studies, naturalistic stimuli, computational models, and sophisticated analytic approaches for characterizing the dynamics of brain activity. These methods will determine the conditions under which the visual system generates hierarchical anticipatory signals, the content and flexibility of those signals, how they arise, and their consequences for behavior. Aim 1 will establish how hierarchical anticipatory signals form and what they represent. We hypothesize that such hierarchical anticipation depends on input from memory systems, is informed by pre-existing schema, and is flexible in the visual features and timescales represented. Aim 2 will determine how hierarchical anticipatory signals may be affected by top- down goals to simulate the future, and how these signals relate to future-oriented visual behavior at a range of timescales. We hypothesize that the visual hierarchy differentially updates its anticipatory signals when the environment or goals change, and generates predictive signals in novel situations by linking separate episodic memories. Finally, Aim 3 will test competing theories of the structure of anticipatory representations. We hypothesize that anticipatory representations are influenced by both temporal and semantic relationships within an event sequence, and propose a computational model for predicting anticipatory event representations learned from a temporally-structured stimulus. Together, the findings will elucidate the mechanisms by which the visual system forms and flexibly updates anticipatory representations at multiple timescales, and how these representations relate to anticipatory behavior in naturalistic conditions. Such insights are important because expectations are instrumental in allowing individuals to behave adaptively, and disruption of visual anticipation might broadly impair goal-directed behavior. This work will therefore shed light on how the capacity to anticipate upcoming events to adaptively guide behavior might be impaired following damage to different parts of the visual system, including higher-order areas whose damage is not associated with primary visual deficits. Together, the results will provide empirical tests of the structure and development of anticipatory signals across the visual hierarchy, informing theories that consider the brain to be a fundamentally predictive organ.