PredictiveBrain · Predicting the future: How our brain predicts future events for successful navigation through our dynamic environment.

Predicting the future: How our brain predicts future events for successful navigation through our dynamic environment.

To successfully navigate our dynamic environment, our brain needs to continuously update its representation of external information. This poses a fundamental problem: how does the brain cope with a stream of dynamic input? It takes time to transmit and process information along the hierarchy of the visual system. Our capacity to interact with dynamic stimuli in a timely manner (e.g., catch a ball) suggests that our brain generates predictions of unfolding dynamics. Without such predictions, there would be a substantial time lag between states in the external world, our perception of these states, and our subsequent reactions. This notion has been supported by recent empirical evidence, e.g., for the prediction of the future position of moving dots in early visual cortex (V1). However, this approach is restricted to low-level visual information that can be represented retinotopically. Hence, while providing clear evidence for a neural prediction mechanism, it only targets the outcome of this process in early visual cortex. The temporal dynamics and representational nature of predictions, in particular at higher levels of processing, remain a largely unexplored research topic. One approach for investigating neural representations is representational similarity analysis (RSA), which typically uses models of static stimulus features at different hierarchical levels of complexity (e.g., color, shape, category, concept) to investigate how these features are represented in the brain. Before action onset I developed a novel powerful and versatile dynamic extension to RSA that allows for the first time to quantify exactly what our brain represents when in naturalistic continuous input such as movies, speech and music. In short, dRSA allows quantifying precisely how strong the match is between continuous representations in the brain and a continuous feature of naturalistic dynamic stimuli. Besides the veridicality (strength of match) of representations, it also quantifies at millisecond accuracy how representations temporally relate to (follow or precede) actual events. In other words, it captures exactly what our brain represents when in naturalistic continuous input. Veridicality and latency of neural representations are reflected in dRSA latency plots in the peak amplitude and peak latency, respectively (Fig. 1). In case of feedforward processing, one expects a lag between the model and the best-matching neural representation (i.e., time needed for information to pass from retina to V1), as reflected in a peak to the right of the vertical zero-lag midline (e.g., pixelwise luminance in Fig. 1a). Prediction should reduce this lag, potentially to it being negative (i.e., left of zero-lag midline), in which case representational content predicts the future model state. The overall objective of the action was to use this approach to characterise the representational dynamics of our brain under naturalistic continuous conditions.

Data: CORDIS, © European Union