Exploring Predictive Processing and Predictive Coding in Cognitive Neuroscience

Question 1

It is difficult to find a contemporary paper in cognitive neuroscience that does not defer to the notion of predictive processing and associated schemes like predictive coding that implement predictive processing in cortical and subcortical hierarchies. For people not familiar with the rhetoric in this field, it is useful to distinguish between the principles of predictive processing and the neuronal process theories that might implement them. There are 2 approaches to predictive processing:

The low road usually starts from kantian notions and helmholtz’s formulation of perception as unconscious inference. The basic idea is that the brain is a constructive organ, actively generating explanations for the sensorium and then testing its hypotheses against sensory data. This notion underwrites predictive coding in the brain, a scheme originally developed to compress sound files in the 1950s. Predictive coding is appealing in its simplicity: Essentially, it sets up a number of competing expectations about the causes of sensory input and then revises or updates these expectations on the basis of prediction errors. These errors are just the difference between what was predicted and what is actually observed.

What would we expect to measure if predictive coding was the right kind of theory?

The ensuing belief updating can then be expressed as a recursive exchange of signals between neuronal populations encoding expected states of the world generating sensations and prediction errors. When predictions are generated under a hierarchical (generative) model of how (hidden) states of the world cause other states, we have a message-passing scheme that looks very much like the recurrent exchange of signals in visual cortical hierarchies; with ascending (prediction error) connections and a descending (prediction) counter-stream (fig 1). Predictive processing has emerged as an enactive generalisation plos biology. This is an open access article distributed under the terms of the creative commons attribution license, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: Kf is a wellcome principal research fellow. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

What do we need separate neuronal populations to encode positive and negative prediction errors ?

The author has declared that no competing interests exist. Abbreviations: Eeg, electroencephalographic. Provenance: Commissioned; externally peerreviewed. Fig 1. This figure illustrates the basic architecture of message passing in hierarchical predictive coding. Here, 3 hierarchical levels are shown in schematic form (blue boxes). These levels are populated by pairs of populations—encoding prediction errors (red triangles) and expectations (blue triangles). Sensory input enters at the lowest level of the hierarchy (denoted by the eye). The equations describe the mathematical form of predictive coding, expressed as a kalman-bucy filter. Here, μ represents an expectation (i.e., mean) of some state of the world. Conversely, ε represents prediction error, which is just the difference between sensory input s and predictions of that input under some generative model g, given expected states of affairs. This form means that prediction errors are weighted by their precision p to drive expectations, which, in turn, supply predictions to error units, thereby suppressing them.

Question 2

The architecture on the left is the canonical or standard architecture, in which prediction errors ascend from one level to the next and are complemented by a descending counter-stream of predictions. However, exactly the same message passing can be implemented by simply moving the expectation units down a little (depicted by the blue arrow) to produce the architecture on the right. Although nothing changes in terms of inference, now the predictions ascend the hierarchy, while prediction errors are conveyed by extrinsic (between cortical level) connections. The self-connections (in blue) stand in for a precision or gain control that modulates the disinhibition of error units. The precision of prediction errors at different hierarchical levels can have profound effects on message passing and subsequent belief updating or evidence accumulation of this idea to encompass action (through the introduction of motor and autonomic reflexes) and applying the underlying (bayesian) principles to planning and policy selection. An alternative (high road) starts with the variational principles that underwrite self-organisation and assembly in sentient systems, to show that such systems can always be cast as making bayesian inferences about their world, i.e., self-evidencing.

What would predictive coding predict about oscillations in the brain? There are a few rules with which all variants of predictive coding must comply.

This leads to the notion of active inference, which can be regarded as a first principle account of predictive processing [8]. The key thing here is that active inference and predictive processing inherit from first principles, whereas schemes like predictive coding are particular (neuronal) process theories about how these principles are manifested in the brain. This means that all the heavy lifting—in terms of asking the right empirical questions—pertains to how predictive processing is implemented.

At present, predictive coding is the prime candidate for predictive processing, largely in virtue of its remarkable power in explaining extrinsic hierarchical connections and the intrinsic connectivity of cortical microcircuits. However, predictive coding is just a theory, and there are an increasing number of variants. Indeed, there are little ‘representation wars’ within the field. For example, or these issues matter when it comes to finding definitive empirical evidence for the computational architectures entailed by predictive coding. 

Why are prediction errors or predictions passed forward from a lower level to a higher level? 

Predictions of predictive coding there are many aspects of functional architectures and neurophysiology that might underwrite predictive coding. We will focus on electrophysiological oscillations. These include the existence of separable neurons or neuronal populations encoding expected states of the world and prediction errors. Crucially, for every expectation unit there is an accompanying error unit. Furthermore, expectation units only provide afferents to error units and vice versa. Furthermore, these recurrent connections must possess the form of a negative feedback loop. This holds irrespective of whether the reciprocal connections are intrinsic (within a cortical level) or extrinsic (linking different hierarchical levels).